blake3 0.2 → 0.3

raw patch · 24 files changed

+5755/−454 lines, 24 filesPVP ok

version bump matches the API change (PVP)

API changes (from Hackage documentation)

Files

• CHANGELOG.md +29/−0
• README.md +1/−1
• blake3.cabal +19/−3
• cbits/README.md +141/−46
• cbits/blake3.c +25/−7
• cbits/blake3.h +5/−1
• cbits/blake3_avx2.c +15/−17
• cbits/blake3_avx2_x86-64_unix.S +15/−0
• cbits/blake3_avx2_x86-64_windows_gnu.S +1/−1
• cbits/blake3_avx512.c +37/−51
• cbits/blake3_avx512_x86-64_unix.S +17/−1
• cbits/blake3_avx512_x86-64_windows_gnu.S +1/−1
• cbits/blake3_dispatch.c +36/−5
• cbits/blake3_impl.h +56/−21
• cbits/blake3_portable.c +1/−9
• cbits/blake3_sse2.c +577/−0
• cbits/blake3_sse2_x86-64_unix.S +2291/−0
• cbits/blake3_sse2_x86-64_windows_gnu.S +2332/−0
• cbits/blake3_sse41.c +15/−12
• cbits/blake3_sse41_x86-64_unix.S +17/−0
• cbits/blake3_sse41_x86-64_windows_gnu.S +1/−1
• lib/BLAKE3.hs +47/−72
• lib/BLAKE3/IO.hs +48/−157
• test/Main.hs +28/−48

@@ -1,3 +1,32 @@+# Version 0.3++* COMPILER ASSISTED BACKWARDS INCOMPATIBLE CHANGE: The `hash` function now takes+  an optional `Key`.++* COMPILER ASSISTED BACKWARDS INCOMPATIBLE CHANGE: The `hasher` function was+  removed. Instead, use `init` without specifying a `Key`.++* COMPILER ASSISTED BACKWARDS INCOMPATIBLE CHANGE: The `hashKeyed` and+  `hasherKeyed` functions were removed. Instead, specify the `Key` when using+  `hash` and `init`.++* COMPILER ASSISTED BACKWARDS INCOMPATIBLE CHANGE: The `digest` functions is not+  exported anymore. The `Digest` constructor is exported instead.++* COMPILER ASSISTED BACKWARDS INCOMPATIBLE CHANGE: The `Context` datatype and the+  `context` function are not exported anymore. The `derive` function takes a+  polymorphic context instead.++* Functions that previously returned a `Digest` now return a polymorphic+  `ByteArrayN`. This makes it easy for downstream libraries to reuse any BLAKE3+  output for other purposes without having to copy bytes over. The `Digest`+  datatype is still exported as a convenience.++* Added `sse2` Cabal flag to enable SSE2 optimizations.++* Bumped upstream BLAKE3 C sources to latest version as of February 4, 2023.++ # Version 0.2  * COMPILER ASSISTED BACKWARDS INCOMPATIBLE CHANGE: Drop the `BLAKE3.Raw` module

@@ -2,7 +2,7 @@  Bindings to the [official fast BLAKE3 implementations in assembly and C](https://github.com/BLAKE3-team/BLAKE3),-with support for AVX-512, AVX2 and SSE 4.1.+with support for AVX-512, AVX2 and SSE 4.1, SSE 2.  # Development 

@@ -1,6 +1,6 @@ cabal-version: 2.2 name: blake3-version: 0.2+version: 0.3 license: Apache-2.0 license-file: LICENSE extra-source-files: README.md CHANGELOG.md@@ -11,10 +11,10 @@ build-type: Simple synopsis: BLAKE3 hashing algorithm description: Bindings to the official fast BLAKE3 implementations in assembly-             and C, with support for AVX-512, AVX2 and SSE 4.1.+             and C, with support for AVX-512, AVX2, SSE 2, and SSE 4.1. homepage: https://github.com/k0001/hs-blake3 bug-reports: https://github.com/k0001/hs-blake3/issues-tested-with: GHC == 8.6.5, GHC == 8.8.3+tested-with: GHC == 9.2.5 extra-source-files:   cbits/*.h   cbits/README.md@@ -35,6 +35,11 @@   default: True   manual: True +flag sse2+  description: Enable SSE 2 instructions+  default: True+  manual: True+ common basic   default-language: Haskell2010   ghc-options: -Wall -Werror=incomplete-patterns@@ -87,6 +92,17 @@       c-sources: cbits/blake3_sse41.c   else     cc-options: -DBLAKE3_NO_SSE41++  -- SSE 2 support+  if flag(sse2)+    if arch(x86_64) && (os(linux) || os(darwin))+      c-sources: cbits/blake3_sse2_x86-64_unix.S+    elif arch(x86_64) && os(windows)+      c-sources: cbits/blake3_sse2_x86-64_windows_gnu.S+    else+      c-sources: cbits/blake3_sse2.c+  else+    cc-options: -DBLAKE3_NO_SSE2   test-suite test

@@ -7,19 +7,29 @@  ```c #include "blake3.h"+#include <errno.h> #include <stdio.h>+#include <stdlib.h>+#include <string.h> #include <unistd.h> -int main() {+int main(void) {   // Initialize the hasher.   blake3_hasher hasher;   blake3_hasher_init(&hasher);    // Read input bytes from stdin.   unsigned char buf[65536];-  ssize_t n;-  while ((n = read(STDIN_FILENO, buf, sizeof(buf))) > 0) {-    blake3_hasher_update(&hasher, buf, n);+  while (1) {+    ssize_t n = read(STDIN_FILENO, buf, sizeof(buf));+    if (n > 0) {+      blake3_hasher_update(&hasher, buf, n);+    } else if (n == 0) {+      break; // end of file+    } else {+      fprintf(stderr, "read failed: %s\n", strerror(errno));+      exit(1);+    }   }    // Finalize the hash. BLAKE3_OUT_LEN is the default output length, 32 bytes.@@ -35,12 +45,14 @@ } ``` -If you save the example code above as `example.c`, and you're on x86\_64-with a Unix-like OS, you can compile a working binary like this:+The code above is included in this directory as `example.c`. If you're+on x86\_64 with a Unix-like OS, you can compile a working binary like+this:  ```bash gcc -O3 -o example example.c blake3.c blake3_dispatch.c blake3_portable.c \-    blake3_sse41_x86-64_unix.S blake3_avx2_x86-64_unix.S blake3_avx512_x86-64_unix.S+    blake3_sse2_x86-64_unix.S blake3_sse41_x86-64_unix.S blake3_avx2_x86-64_unix.S \+    blake3_avx512_x86-64_unix.S ```  # API@@ -70,6 +82,8 @@  Initialize a `blake3_hasher` in the default hashing mode. +---+ ```c void blake3_hasher_update(   blake3_hasher *self,@@ -79,6 +93,8 @@  Add input to the hasher. This can be called any number of times. +---+ ```c void blake3_hasher_finalize(   const blake3_hasher *self,@@ -86,10 +102,11 @@   size_t out_len); ``` -Finalize the hasher and emit an output of any length. This doesn't-modify the hasher itself, and it's possible to finalize again after-adding more input. The constant `BLAKE3_OUT_LEN` provides the default-output length, 32 bytes.+Finalize the hasher and return an output of any length, given in bytes.+This doesn't modify the hasher itself, and it's possible to finalize+again after adding more input. The constant `BLAKE3_OUT_LEN` provides+the default output length, 32 bytes, which is recommended for most+callers. See the [Security Notes](#security-notes) below.  ## Less Common API Functions @@ -102,22 +119,50 @@ Initialize a `blake3_hasher` in the keyed hashing mode. The key must be exactly 32 bytes. +---+ ```c void blake3_hasher_init_derive_key(   blake3_hasher *self,   const char *context); ``` -Initialize a `blake3_hasher` in the key derivation mode. Key material-should be given as input after initialization, using-`blake3_hasher_update`. `context` is a standard C string of any length,-and the terminating null byte is not included. The context string should-be hardcoded, globally unique, and application-specific. A good default-format for the context string is `"[application] [commit timestamp]-[purpose]"`, e.g., `"example.com 2019-12-25 16:18:03 session tokens-v1"`.+Initialize a `blake3_hasher` in the key derivation mode. The context+string is given as an initialization parameter, and afterwards input key+material should be given with `blake3_hasher_update`. The context string+is a null-terminated C string which should be **hardcoded, globally+unique, and application-specific**. The context string should not+include any dynamic input like salts, nonces, or identifiers read from a+database at runtime. A good default format for the context string is+`"[application] [commit timestamp] [purpose]"`, e.g., `"example.com+2019-12-25 16:18:03 session tokens v1"`. +This function is intended for application code written in C. For+language bindings, see `blake3_hasher_init_derive_key_raw` below.++---+ ```c+void blake3_hasher_init_derive_key_raw(+  blake3_hasher *self,+  const void *context,+  size_t context_len);+```++As `blake3_hasher_init_derive_key` above, except that the context string+is given as a pointer to an array of arbitrary bytes with a provided+length. This is intended for writing language bindings, where C string+conversion would add unnecessary overhead and new error cases. Unicode+strings should be encoded as UTF-8.++Application code in C should prefer `blake3_hasher_init_derive_key`,+which takes the context as a C string. If you need to use arbitrary+bytes as a context string in application code, consider whether you're+violating the requirement that context strings should be hardcoded.++---++```c void blake3_hasher_finalize_seek(   const blake3_hasher *self,   uint64_t seek,@@ -130,6 +175,36 @@ efficiently stream a large output without allocating memory, call this function in a loop, incrementing `seek` by the output length each time. +---++```c+void blake3_hasher_reset(+  blake3_hasher *self);+```++Reset the hasher to its initial state, prior to any calls to+`blake3_hasher_update`. Currently this is no different from calling+`blake3_hasher_init` or similar again. However, if this implementation gains+multithreading support in the future, and if `blake3_hasher` holds (optional)+threading resources, this function will reuse those resources. Until then, this+is mainly for feature compatibility with the Rust implementation.++# Security Notes++Outputs shorter than the default length of 32 bytes (256 bits) provide less security. An N-bit+BLAKE3 output is intended to provide N bits of first and second preimage resistance and N/2+bits of collision resistance, for any N up to 256. Longer outputs don't provide any additional+security.++Avoid relying on the secrecy of the output offset, that is, the `seek` argument of+`blake3_hasher_finalize_seek`. [_Block-Cipher-Based Tree Hashing_ by Aldo+Gunsing](https://eprint.iacr.org/2022/283) shows that an attacker who knows both the message+and the key (if any) can easily determine the offset of an extended output. For comparison,+AES-CTR has a similar property: if you know the key, you can decrypt a block from an unknown+position in the output stream to recover its block index. Callers with strong secret keys+aren't affected in practice, but secret offsets are a [design+smell](https://en.wikipedia.org/wiki/Design_smell) in any case.+ # Building  This implementation is just C and assembly files. It doesn't include a@@ -144,13 +219,13 @@ Dynamic dispatch is enabled by default on x86. The implementation will query the CPU at runtime to detect SIMD support, and it will use the widest instruction set available. By default, `blake3_dispatch.c`-expects to be linked with code for four different instruction sets:-portable C, SSE4.1, AVX2, and AVX-512.+expects to be linked with code for five different instruction sets:+portable C, SSE2, SSE4.1, AVX2, and AVX-512. -For each of the x86 SIMD instruction sets, two versions are available,-one in assembly (with three flavors: Unix, Windows MSVC, and Windows-GNU) and one using C intrinsics. The assembly versions are generally-preferred: they perform better, they perform more consistently across+For each of the x86 SIMD instruction sets, four versions are available:+three flavors of assembly (Unix, Windows MSVC, and Windows GNU) and one+version using C intrinsics. The assembly versions are generally+preferred. They perform better, they perform more consistently across different compilers, and they build more quickly. On the other hand, the assembly versions are x86\_64-only, and you need to select the right flavor for your target platform.@@ -160,40 +235,59 @@  ```bash gcc -shared -O3 -o libblake3.so blake3.c blake3_dispatch.c blake3_portable.c \-    blake3_sse41_x86-64_unix.S blake3_avx2_x86-64_unix.S blake3_avx512_x86-64_unix.S+    blake3_sse2_x86-64_unix.S blake3_sse41_x86-64_unix.S blake3_avx2_x86-64_unix.S \+    blake3_avx512_x86-64_unix.S ``` -Here's the same shared library using the intrinsics-based implementations:+When building the intrinsics-based implementations, you need to build+each implementation separately, with the corresponding instruction set+explicitly enabled in the compiler. Here's the same shared library using+the intrinsics-based implementations:  ```bash+gcc -c -fPIC -O3 -msse2 blake3_sse2.c -o blake3_sse2.o+gcc -c -fPIC -O3 -msse4.1 blake3_sse41.c -o blake3_sse41.o+gcc -c -fPIC -O3 -mavx2 blake3_avx2.c -o blake3_avx2.o+gcc -c -fPIC -O3 -mavx512f -mavx512vl blake3_avx512.c -o blake3_avx512.o gcc -shared -O3 -o libblake3.so blake3.c blake3_dispatch.c blake3_portable.c \-    blake3_avx2.c blake3_avx512.c blake3_sse41.c+    blake3_avx2.o blake3_avx512.o blake3_sse41.o blake3_sse2.o ``` -When building the intrinsics-based implementations under MSVC, you need to-build `blake3_avx2.c` and `blake3_avx512.c` separately first, specifying the-`/arch:AVX2` and `/arch:AVX512` compiler flags respectively.+Note above that building `blake3_avx512.c` requires both `-mavx512f` and+`-mavx512vl` under GCC and Clang. Under MSVC, the single `/arch:AVX512`+flag is sufficient. The MSVC equivalent of `-mavx2` is `/arch:AVX2`.+MSVC enables SSE2 and SSE4.1 by defaut, and it doesn't have a+corresponding flag. -If you want to omit SIMD code on x86, you need to explicitly disable+If you want to omit SIMD code entirely, you need to explicitly disable each instruction set. Here's an example of building a shared library on x86 with only portable code:  ```bash-gcc -shared -O3 -o libblake3.so -DBLAKE3_NO_SSE41 -DBLAKE3_NO_AVX2 -DBLAKE3_NO_AVX512 \-    blake3.c blake3_dispatch.c blake3_portable.c+gcc -shared -O3 -o libblake3.so -DBLAKE3_NO_SSE2 -DBLAKE3_NO_SSE41 -DBLAKE3_NO_AVX2 \+    -DBLAKE3_NO_AVX512 blake3.c blake3_dispatch.c blake3_portable.c ```  ## ARM NEON -The NEON implementation is not enabled by default on ARM, since not all-ARM targets support it. To enable it, set `BLAKE3_USE_NEON=1`. Here's an-example of building a shared library on ARM Linux with NEON support:+The NEON implementation is enabled by default on AArch64, but not on+other ARM targets, since not all of them support it. To enable it, set+`BLAKE3_USE_NEON=1`. Here's an example of building a shared library on+ARM Linux with NEON support:  ```bash-gcc -shared -O3 -o libblake3.so -DBLAKE3_USE_NEON blake3.c blake3_dispatch.c \+gcc -shared -O3 -o libblake3.so -DBLAKE3_USE_NEON=1 blake3.c blake3_dispatch.c \     blake3_portable.c blake3_neon.c ``` +To explicitiy disable using NEON instructions on AArch64, set+`BLAKE3_USE_NEON=0`.++```bash+gcc -shared -O3 -o libblake3.so -DBLAKE3_USE_NEON=0 blake3.c blake3_dispatch.c \+    blake3_portable.c+```+ Note that on some targets (ARMv7 in particular), extra flags may be required to activate NEON support in the compiler. If you see an error like...@@ -215,12 +309,13 @@ gcc -shared -O3 -o libblake3.so blake3.c blake3_dispatch.c blake3_portable.c ``` -# Differences from the Rust Implementation--The single-threaded Rust and C implementations use the same algorithms,-and their performance is the same if you use the assembly-implementations or if you compile the intrinsics-based implementations-with Clang. (Both Clang and rustc are LLVM-based.)+# Multithreading -The C implementation doesn't currently support multi-threading. OpenMP-support or similar might be added in the future.+Unlike the Rust implementation, the C implementation doesn't currently support+multithreading. A future version of this library could add support by taking an+optional dependency on OpenMP or similar. Alternatively, we could expose a+lower-level API to allow callers to implement concurrency themselves. The+former would be more convenient and less error-prone, but the latter would give+callers the maximum possible amount of control. The best choice here depends on+the specific use case, so if you have a use case for multithreaded hashing in+C, please file a GitHub issue and let us know.

@@ -5,6 +5,8 @@ #include "blake3.h" #include "blake3_impl.h" +const char *blake3_version(void) { return BLAKE3_VERSION_STRING; }+ INLINE void chunk_state_init(blake3_chunk_state *self, const uint32_t key[8],                              uint8_t flags) {   memcpy(self->cv, key, BLAKE3_KEY_LEN);@@ -81,7 +83,7 @@   memcpy(cv_words, self->input_cv, 32);   blake3_compress_in_place(cv_words, self->block, self->block_len,                            self->counter, self->flags);-  memcpy(cv, cv_words, 32);+  store_cv_words(cv, cv_words); }  INLINE void output_root_bytes(const output_t *self, uint64_t seek, uint8_t *out,@@ -244,7 +246,7 @@  // The wide helper function returns (writes out) an array of chaining values // and returns the length of that array. The number of chaining values returned-// is the dyanmically detected SIMD degree, at most MAX_SIMD_DEGREE. Or fewer,+// is the dynamically detected SIMD degree, at most MAX_SIMD_DEGREE. Or fewer, // if the input is shorter than that many chunks. The reason for maintaining a // wide array of chaining values going back up the tree, is to allow the // implementation to hash as many parents in parallel as possible.@@ -252,7 +254,7 @@ // As a special case when the SIMD degree is 1, this function will still return // at least 2 outputs. This guarantees that this function doesn't perform the // root compression. (If it did, it would use the wrong flags, and also we-// wouldn't be able to implement exendable ouput.) Note that this function is+// wouldn't be able to implement exendable output.) Note that this function is // not used when the whole input is only 1 chunk long; that's a different // codepath. //@@ -335,15 +337,21 @@   assert(input_len > BLAKE3_CHUNK_LEN); #endif -  uint8_t cv_array[2 * MAX_SIMD_DEGREE_OR_2 * BLAKE3_OUT_LEN];+  uint8_t cv_array[MAX_SIMD_DEGREE_OR_2 * BLAKE3_OUT_LEN];   size_t num_cvs = blake3_compress_subtree_wide(input, input_len, key,                                                 chunk_counter, flags, cv_array);+  assert(num_cvs <= MAX_SIMD_DEGREE_OR_2);    // If MAX_SIMD_DEGREE is greater than 2 and there's enough input,   // compress_subtree_wide() returns more than 2 chaining values. Condense   // them into 2 by forming parent nodes repeatedly.   uint8_t out_array[MAX_SIMD_DEGREE_OR_2 * BLAKE3_OUT_LEN / 2];-  while (num_cvs > 2) {+  // The second half of this loop condition is always true, and we just+  // asserted it above. But GCC can't tell that it's always true, and if NDEBUG+  // is set on platforms where MAX_SIMD_DEGREE_OR_2 == 2, GCC emits spurious+  // warnings here. GCC 8.5 is particularly sensitive, so if you're changing+  // this code, test it against that version.+  while (num_cvs > 2 && num_cvs <= MAX_SIMD_DEGREE_OR_2) {     num_cvs =         compress_parents_parallel(cv_array, num_cvs, key, flags, out_array);     memcpy(cv_array, out_array, num_cvs * BLAKE3_OUT_LEN);@@ -367,10 +375,11 @@   hasher_init_base(self, key_words, KEYED_HASH); } -void blake3_hasher_init_derive_key(blake3_hasher *self, const char *context) {+void blake3_hasher_init_derive_key_raw(blake3_hasher *self, const void *context,+                                       size_t context_len) {   blake3_hasher context_hasher;   hasher_init_base(&context_hasher, IV, DERIVE_KEY_CONTEXT);-  blake3_hasher_update(&context_hasher, context, strlen(context));+  blake3_hasher_update(&context_hasher, context, context_len);   uint8_t context_key[BLAKE3_KEY_LEN];   blake3_hasher_finalize(&context_hasher, context_key, BLAKE3_KEY_LEN);   uint32_t context_key_words[8];@@ -378,6 +387,10 @@   hasher_init_base(self, context_key_words, DERIVE_KEY_MATERIAL); } +void blake3_hasher_init_derive_key(blake3_hasher *self, const char *context) {+  blake3_hasher_init_derive_key_raw(self, context, strlen(context));+}+ // As described in hasher_push_cv() below, we do "lazy merging", delaying // merges until right before the next CV is about to be added. This is // different from the reference implementation. Another difference is that we@@ -595,4 +608,9 @@     output = parent_output(parent_block, self->key, self->chunk.flags);   }   output_root_bytes(&output, seek, out, out_len);+}++void blake3_hasher_reset(blake3_hasher *self) {+  chunk_state_reset(&self->chunk, self->key, 0);+  self->cv_stack_len = 0; }

@@ -8,12 +8,12 @@ extern "C" { #endif +#define BLAKE3_VERSION_STRING "1.3.3" #define BLAKE3_KEY_LEN 32 #define BLAKE3_OUT_LEN 32 #define BLAKE3_BLOCK_LEN 64 #define BLAKE3_CHUNK_LEN 1024 #define BLAKE3_MAX_DEPTH 54-#define BLAKE3_MAX_SIMD_DEGREE 16  // This struct is a private implementation detail. It has to be here because // it's part of blake3_hasher below.@@ -38,16 +38,20 @@   uint8_t cv_stack[(BLAKE3_MAX_DEPTH + 1) * BLAKE3_OUT_LEN]; } blake3_hasher; +const char *blake3_version(void); void blake3_hasher_init(blake3_hasher *self); void blake3_hasher_init_keyed(blake3_hasher *self,                               const uint8_t key[BLAKE3_KEY_LEN]); void blake3_hasher_init_derive_key(blake3_hasher *self, const char *context);+void blake3_hasher_init_derive_key_raw(blake3_hasher *self, const void *context,+                                       size_t context_len); void blake3_hasher_update(blake3_hasher *self, const void *input,                           size_t input_len); void blake3_hasher_finalize(const blake3_hasher *self, uint8_t *out,                             size_t out_len); void blake3_hasher_finalize_seek(const blake3_hasher *self, uint64_t seek,                                  uint8_t *out, size_t out_len);+void blake3_hasher_reset(blake3_hasher *self);  #ifdef __cplusplus }

@@ -2,53 +2,49 @@  #include <immintrin.h> +#if defined(__clang__)+#pragma clang attribute push (__attribute__((target("avx2"))), apply_to=function)+#elif defined(__GNUC__)+#pragma GCC target("avx2")+#endif+ #define DEGREE 8 -TARGET_AVX2 INLINE __m256i loadu(const uint8_t src[32]) {   return _mm256_loadu_si256((const __m256i *)src); } -TARGET_AVX2 INLINE void storeu(__m256i src, uint8_t dest[16]) {   _mm256_storeu_si256((__m256i *)dest, src); } -TARGET_AVX2 INLINE __m256i addv(__m256i a, __m256i b) { return _mm256_add_epi32(a, b); }  // Note that clang-format doesn't like the name "xor" for some reason.-TARGET_AVX2 INLINE __m256i xorv(__m256i a, __m256i b) { return _mm256_xor_si256(a, b); } -TARGET_AVX2 INLINE __m256i set1(uint32_t x) { return _mm256_set1_epi32((int32_t)x); } -TARGET_AVX2 INLINE __m256i rot16(__m256i x) {   return _mm256_shuffle_epi8(       x, _mm256_set_epi8(13, 12, 15, 14, 9, 8, 11, 10, 5, 4, 7, 6, 1, 0, 3, 2,                          13, 12, 15, 14, 9, 8, 11, 10, 5, 4, 7, 6, 1, 0, 3, 2)); } -TARGET_AVX2 INLINE __m256i rot12(__m256i x) {   return _mm256_or_si256(_mm256_srli_epi32(x, 12), _mm256_slli_epi32(x, 32 - 12)); } -TARGET_AVX2 INLINE __m256i rot8(__m256i x) {   return _mm256_shuffle_epi8(       x, _mm256_set_epi8(12, 15, 14, 13, 8, 11, 10, 9, 4, 7, 6, 5, 0, 3, 2, 1,                          12, 15, 14, 13, 8, 11, 10, 9, 4, 7, 6, 5, 0, 3, 2, 1)); } -TARGET_AVX2 INLINE __m256i rot7(__m256i x) {   return _mm256_or_si256(_mm256_srli_epi32(x, 7), _mm256_slli_epi32(x, 32 - 7)); } -TARGET_AVX2 INLINE void round_fn(__m256i v[16], __m256i m[16], size_t r) {   v[0] = addv(v[0], m[(size_t)MSG_SCHEDULE[r][0]]);   v[1] = addv(v[1], m[(size_t)MSG_SCHEDULE[r][2]]);@@ -165,7 +161,6 @@   v[4] = rot7(v[4]); } -TARGET_AVX2 INLINE void transpose_vecs(__m256i vecs[DEGREE]) {   // Interleave 32-bit lanes. The low unpack is lanes 00/11/44/55, and the high   // is 22/33/66/77.@@ -200,7 +195,6 @@   vecs[7] = _mm256_permute2x128_si256(abcd_37, efgh_37, 0x31); } -TARGET_AVX2 INLINE void transpose_msg_vecs(const uint8_t *const *inputs,                                size_t block_offset, __m256i out[16]) {   out[0] = loadu(&inputs[0][block_offset + 0 * sizeof(__m256i)]);@@ -220,27 +214,26 @@   out[14] = loadu(&inputs[6][block_offset + 1 * sizeof(__m256i)]);   out[15] = loadu(&inputs[7][block_offset + 1 * sizeof(__m256i)]);   for (size_t i = 0; i < 8; ++i) {-    _mm_prefetch(&inputs[i][block_offset + 256], _MM_HINT_T0);+    _mm_prefetch((const void *)&inputs[i][block_offset + 256], _MM_HINT_T0);   }   transpose_vecs(&out[0]);   transpose_vecs(&out[8]); } -TARGET_AVX2 INLINE void load_counters(uint64_t counter, bool increment_counter,                           __m256i *out_lo, __m256i *out_hi) {   const __m256i mask = _mm256_set1_epi32(-(int32_t)increment_counter);   const __m256i add0 = _mm256_set_epi32(7, 6, 5, 4, 3, 2, 1, 0);   const __m256i add1 = _mm256_and_si256(mask, add0);-  __m256i l = _mm256_add_epi32(_mm256_set1_epi32(counter), add1);+  __m256i l = _mm256_add_epi32(_mm256_set1_epi32((int32_t)counter), add1);   __m256i carry = _mm256_cmpgt_epi32(_mm256_xor_si256(add1, _mm256_set1_epi32(0x80000000)),                                      _mm256_xor_si256(   l, _mm256_set1_epi32(0x80000000)));-  __m256i h = _mm256_sub_epi32(_mm256_set1_epi32(counter >> 32), carry);+  __m256i h = _mm256_sub_epi32(_mm256_set1_epi32((int32_t)(counter >> 32)), carry);   *out_lo = l;   *out_hi = h; } -TARGET_AVX2+static void blake3_hash8_avx2(const uint8_t *const *inputs, size_t blocks,                        const uint32_t key[8], uint64_t counter,                        bool increment_counter, uint8_t flags,@@ -337,3 +330,8 @@                             out); #endif }++#if defined(__clang__)+#pragma clang attribute pop+#endif+

@@ -1,3 +1,17 @@+#if defined(__ELF__) && defined(__linux__)+.section .note.GNU-stack,"",%progbits+#endif++#if defined(__ELF__) && defined(__CET__) && defined(__has_include)+#if __has_include(<cet.h>)+#include <cet.h>+#endif+#endif++#if !defined(_CET_ENDBR)+#define _CET_ENDBR+#endif+ .intel_syntax noprefix .global _blake3_hash_many_avx2 .global blake3_hash_many_avx2@@ -9,6 +23,7 @@         .p2align  6 _blake3_hash_many_avx2: blake3_hash_many_avx2:+        _CET_ENDBR         push    r15         push    r14         push    r13

@@ -1784,7 +1784,7 @@         vmovdqu xmmword ptr [rbx+0x10], xmm1         jmp     4b -.section .rodata+.section .rdata .p2align  6 ADD0:         .long  0, 1, 2, 3, 4, 5, 6, 7

@@ -2,6 +2,12 @@  #include <immintrin.h> +#if defined(__clang__)+#pragma clang attribute push (__attribute__((target("avx512vl,avx512f"))), apply_to=function)+#elif defined(__GNUC__)+#pragma GCC target("avx512vl,avx512f")+#endif+ #define _mm_shuffle_ps2(a, b, c)                                               \   (_mm_castps_si128(                                                           \       _mm_shuffle_ps(_mm_castsi128_ps(a), _mm_castsi128_ps(b), (c))))@@ -10,12 +16,10 @@   return _mm_loadu_si128((const __m128i *)src); } -TARGET_AVX512 INLINE __m256i loadu_256(const uint8_t src[32]) {   return _mm256_loadu_si256((const __m256i *)src); } -TARGET_AVX512 INLINE __m512i loadu_512(const uint8_t src[64]) {   return _mm512_loadu_si512((const __m512i *)src); }@@ -24,73 +28,54 @@   _mm_storeu_si128((__m128i *)dest, src); } -TARGET_AVX512 INLINE void storeu_256(__m256i src, uint8_t dest[16]) {   _mm256_storeu_si256((__m256i *)dest, src); }  INLINE __m128i add_128(__m128i a, __m128i b) { return _mm_add_epi32(a, b); } -TARGET_AVX512 INLINE __m256i add_256(__m256i a, __m256i b) { return _mm256_add_epi32(a, b); } -TARGET_AVX512 INLINE __m512i add_512(__m512i a, __m512i b) { return _mm512_add_epi32(a, b); }  INLINE __m128i xor_128(__m128i a, __m128i b) { return _mm_xor_si128(a, b); } -TARGET_AVX512 INLINE __m256i xor_256(__m256i a, __m256i b) { return _mm256_xor_si256(a, b); } -TARGET_AVX512 INLINE __m512i xor_512(__m512i a, __m512i b) { return _mm512_xor_si512(a, b); }  INLINE __m128i set1_128(uint32_t x) { return _mm_set1_epi32((int32_t)x); } -TARGET_AVX512 INLINE __m256i set1_256(uint32_t x) { return _mm256_set1_epi32((int32_t)x); } -TARGET_AVX512 INLINE __m512i set1_512(uint32_t x) { return _mm512_set1_epi32((int32_t)x); }  INLINE __m128i set4(uint32_t a, uint32_t b, uint32_t c, uint32_t d) {   return _mm_setr_epi32((int32_t)a, (int32_t)b, (int32_t)c, (int32_t)d); } -TARGET_AVX512 INLINE __m128i rot16_128(__m128i x) { return _mm_ror_epi32(x, 16); } -TARGET_AVX512 INLINE __m256i rot16_256(__m256i x) { return _mm256_ror_epi32(x, 16); } -TARGET_AVX512 INLINE __m512i rot16_512(__m512i x) { return _mm512_ror_epi32(x, 16); } -TARGET_AVX512 INLINE __m128i rot12_128(__m128i x) { return _mm_ror_epi32(x, 12); } -TARGET_AVX512 INLINE __m256i rot12_256(__m256i x) { return _mm256_ror_epi32(x, 12); } -TARGET_AVX512 INLINE __m512i rot12_512(__m512i x) { return _mm512_ror_epi32(x, 12); } -TARGET_AVX512 INLINE __m128i rot8_128(__m128i x) { return _mm_ror_epi32(x, 8); } -TARGET_AVX512 INLINE __m256i rot8_256(__m256i x) { return _mm256_ror_epi32(x, 8); } -TARGET_AVX512 INLINE __m512i rot8_512(__m512i x) { return _mm512_ror_epi32(x, 8); } -TARGET_AVX512 INLINE __m128i rot7_128(__m128i x) { return _mm_ror_epi32(x, 7); } -TARGET_AVX512 INLINE __m256i rot7_256(__m256i x) { return _mm256_ror_epi32(x, 7); } -TARGET_AVX512 INLINE __m512i rot7_512(__m512i x) { return _mm512_ror_epi32(x, 7); }  /*@@ -99,7 +84,6 @@  * ----------------------------------------------------------------------------  */ -TARGET_AVX512 INLINE void g1(__m128i *row0, __m128i *row1, __m128i *row2, __m128i *row3,                __m128i m) {   *row0 = add_128(add_128(*row0, m), *row1);@@ -110,7 +94,6 @@   *row1 = rot12_128(*row1); } -TARGET_AVX512 INLINE void g2(__m128i *row0, __m128i *row1, __m128i *row2, __m128i *row3,                __m128i m) {   *row0 = add_128(add_128(*row0, m), *row1);@@ -136,7 +119,6 @@   *row2 = _mm_shuffle_epi32(*row2, _MM_SHUFFLE(2, 1, 0, 3)); } -TARGET_AVX512 INLINE void compress_pre(__m128i rows[4], const uint32_t cv[8],                          const uint8_t block[BLAKE3_BLOCK_LEN],                          uint8_t block_len, uint64_t counter, uint8_t flags) {@@ -308,7 +290,6 @@   undiagonalize(&rows[0], &rows[2], &rows[3]); } -TARGET_AVX512 void blake3_compress_xof_avx512(const uint32_t cv[8],                                 const uint8_t block[BLAKE3_BLOCK_LEN],                                 uint8_t block_len, uint64_t counter,@@ -321,7 +302,6 @@   storeu_128(xor_128(rows[3], loadu_128((uint8_t *)&cv[4])), &out[48]); } -TARGET_AVX512 void blake3_compress_in_place_avx512(uint32_t cv[8],                                      const uint8_t block[BLAKE3_BLOCK_LEN],                                      uint8_t block_len, uint64_t counter,@@ -338,7 +318,6 @@  * ----------------------------------------------------------------------------  */ -TARGET_AVX512 INLINE void round_fn4(__m128i v[16], __m128i m[16], size_t r) {   v[0] = add_128(v[0], m[(size_t)MSG_SCHEDULE[r][0]]);   v[1] = add_128(v[1], m[(size_t)MSG_SCHEDULE[r][2]]);@@ -495,7 +474,7 @@   out[14] = loadu_128(&inputs[2][block_offset + 3 * sizeof(__m128i)]);   out[15] = loadu_128(&inputs[3][block_offset + 3 * sizeof(__m128i)]);   for (size_t i = 0; i < 4; ++i) {-    _mm_prefetch(&inputs[i][block_offset + 256], _MM_HINT_T0);+    _mm_prefetch((const void *)&inputs[i][block_offset + 256], _MM_HINT_T0);   }   transpose_vecs_128(&out[0]);   transpose_vecs_128(&out[4]);@@ -503,7 +482,6 @@   transpose_vecs_128(&out[12]); } -TARGET_AVX512 INLINE void load_counters4(uint64_t counter, bool increment_counter,                            __m128i *out_lo, __m128i *out_hi) {   uint64_t mask = (increment_counter ? ~0 : 0);@@ -516,7 +494,7 @@   *out_hi = _mm256_cvtepi64_epi32(_mm256_srli_epi64(counters, 32)); } -TARGET_AVX512+static void blake3_hash4_avx512(const uint8_t *const *inputs, size_t blocks,                          const uint32_t key[8], uint64_t counter,                          bool increment_counter, uint8_t flags,@@ -584,7 +562,6 @@  * ----------------------------------------------------------------------------  */ -TARGET_AVX512 INLINE void round_fn8(__m256i v[16], __m256i m[16], size_t r) {   v[0] = add_256(v[0], m[(size_t)MSG_SCHEDULE[r][0]]);   v[1] = add_256(v[1], m[(size_t)MSG_SCHEDULE[r][2]]);@@ -701,7 +678,6 @@   v[4] = rot7_256(v[4]); } -TARGET_AVX512 INLINE void transpose_vecs_256(__m256i vecs[8]) {   // Interleave 32-bit lanes. The low unpack is lanes 00/11/44/55, and the high   // is 22/33/66/77.@@ -736,7 +712,6 @@   vecs[7] = _mm256_permute2x128_si256(abcd_37, efgh_37, 0x31); } -TARGET_AVX512 INLINE void transpose_msg_vecs8(const uint8_t *const *inputs,                                 size_t block_offset, __m256i out[16]) {   out[0] = loadu_256(&inputs[0][block_offset + 0 * sizeof(__m256i)]);@@ -756,13 +731,12 @@   out[14] = loadu_256(&inputs[6][block_offset + 1 * sizeof(__m256i)]);   out[15] = loadu_256(&inputs[7][block_offset + 1 * sizeof(__m256i)]);   for (size_t i = 0; i < 8; ++i) {-    _mm_prefetch(&inputs[i][block_offset + 256], _MM_HINT_T0);+    _mm_prefetch((const void *)&inputs[i][block_offset + 256], _MM_HINT_T0);   }   transpose_vecs_256(&out[0]);   transpose_vecs_256(&out[8]); } -TARGET_AVX512 INLINE void load_counters8(uint64_t counter, bool increment_counter,                            __m256i *out_lo, __m256i *out_hi) {   uint64_t mask = (increment_counter ? ~0 : 0);@@ -775,7 +749,7 @@   *out_hi = _mm512_cvtepi64_epi32(_mm512_srli_epi64(counters, 32)); } -TARGET_AVX512+static void blake3_hash8_avx512(const uint8_t *const *inputs, size_t blocks,                          const uint32_t key[8], uint64_t counter,                          bool increment_counter, uint8_t flags,@@ -840,7 +814,6 @@  * ----------------------------------------------------------------------------  */ -TARGET_AVX512 INLINE void round_fn16(__m512i v[16], __m512i m[16], size_t r) {   v[0] = add_512(v[0], m[(size_t)MSG_SCHEDULE[r][0]]);   v[1] = add_512(v[1], m[(size_t)MSG_SCHEDULE[r][2]]);@@ -960,7 +933,6 @@ // 0b10001000, or lanes a0/a2/b0/b2 in little-endian order #define LO_IMM8 0x88 -TARGET_AVX512 INLINE __m512i unpack_lo_128(__m512i a, __m512i b) {   return _mm512_shuffle_i32x4(a, b, LO_IMM8); }@@ -968,12 +940,10 @@ // 0b11011101, or lanes a1/a3/b1/b3 in little-endian order #define HI_IMM8 0xdd -TARGET_AVX512 INLINE __m512i unpack_hi_128(__m512i a, __m512i b) {   return _mm512_shuffle_i32x4(a, b, HI_IMM8); } -TARGET_AVX512 INLINE void transpose_vecs_512(__m512i vecs[16]) {   // Interleave 32-bit lanes. The _0 unpack is lanes   // 0/0/1/1/4/4/5/5/8/8/9/9/12/12/13/13, and the _2 unpack is lanes@@ -1056,7 +1026,6 @@   vecs[15] = unpack_hi_128(abcdefgh_7, ijklmnop_7); } -TARGET_AVX512 INLINE void transpose_msg_vecs16(const uint8_t *const *inputs,                                  size_t block_offset, __m512i out[16]) {   out[0] = loadu_512(&inputs[0][block_offset]);@@ -1076,25 +1045,37 @@   out[14] = loadu_512(&inputs[14][block_offset]);   out[15] = loadu_512(&inputs[15][block_offset]);   for (size_t i = 0; i < 16; ++i) {-    _mm_prefetch(&inputs[i][block_offset + 256], _MM_HINT_T0);+    _mm_prefetch((const void *)&inputs[i][block_offset + 256], _MM_HINT_T0);   }   transpose_vecs_512(out); } -TARGET_AVX512 INLINE void load_counters16(uint64_t counter, bool increment_counter,                             __m512i *out_lo, __m512i *out_hi) {   const __m512i mask = _mm512_set1_epi32(-(int32_t)increment_counter);-  const __m512i add0 = _mm512_set_epi32(15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0);-  const __m512i add1 = _mm512_and_si512(mask, add0);-  __m512i l = _mm512_add_epi32(_mm512_set1_epi32(counter), add1);-  __mmask16 carry = _mm512_cmp_epu32_mask(l, add1, _MM_CMPINT_LT);-  __m512i h = _mm512_mask_add_epi32(_mm512_set1_epi32(counter >> 32), carry, _mm512_set1_epi32(counter >> 32), _mm512_set1_epi32(1));-  *out_lo = l;-  *out_hi = h;+  const __m512i deltas = _mm512_set_epi32(15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0);+  const __m512i masked_deltas = _mm512_and_si512(deltas, mask);+  const __m512i low_words = _mm512_add_epi32(+    _mm512_set1_epi32((int32_t)counter),+    masked_deltas);+  // The carry bit is 1 if the high bit of the word was 1 before addition and is+  // 0 after.+  // NOTE: It would be a bit more natural to use _mm512_cmp_epu32_mask to+  // compute the carry bits here, and originally we did, but that intrinsic is+  // broken under GCC 5.4. See https://github.com/BLAKE3-team/BLAKE3/issues/271.+  const __m512i carries = _mm512_srli_epi32(+    _mm512_andnot_si512(+        low_words, // 0 after (gets inverted by andnot)+        _mm512_set1_epi32((int32_t)counter)), // and 1 before+    31);+  const __m512i high_words = _mm512_add_epi32(+    _mm512_set1_epi32((int32_t)(counter >> 32)),+    carries);+  *out_lo = low_words;+  *out_hi = high_words; } -TARGET_AVX512+static void blake3_hash16_avx512(const uint8_t *const *inputs, size_t blocks,                           const uint32_t key[8], uint64_t counter,                           bool increment_counter, uint8_t flags,@@ -1243,3 +1224,8 @@     out = &out[BLAKE3_OUT_LEN];   } }++#if defined(__clang__)+#pragma clang attribute pop+#endif+

@@ -1,5 +1,18 @@-.intel_syntax noprefix+#if defined(__ELF__) && defined(__linux__)+.section .note.GNU-stack,"",%progbits+#endif +#if defined(__ELF__) && defined(__CET__) && defined(__has_include)+#if __has_include(<cet.h>)+#include <cet.h>+#endif+#endif++#if !defined(_CET_ENDBR)+#define _CET_ENDBR+#endif++.intel_syntax noprefix .global _blake3_hash_many_avx512 .global blake3_hash_many_avx512 .global blake3_compress_in_place_avx512@@ -15,6 +28,7 @@ .p2align  6 _blake3_hash_many_avx512: blake3_hash_many_avx512:+        _CET_ENDBR         push    r15         push    r14         push    r13@@ -2372,6 +2386,7 @@ .p2align 6 _blake3_compress_in_place_avx512: blake3_compress_in_place_avx512:+        _CET_ENDBR         vmovdqu xmm0, xmmword ptr [rdi]         vmovdqu xmm1, xmmword ptr [rdi+0x10]         movzx   eax, r8b@@ -2454,6 +2469,7 @@ .p2align 6 _blake3_compress_xof_avx512: blake3_compress_xof_avx512:+        _CET_ENDBR         vmovdqu xmm0, xmmword ptr [rdi]         vmovdqu xmm1, xmmword ptr [rdi+0x10]         movzx   eax, r8b

@@ -2587,7 +2587,7 @@         add     rsp, 72         ret -.section .rodata+.section .rdata .p2align  6 INDEX0:         .long    0,  1,  2,  3, 16, 17, 18, 19

@@ -10,12 +10,14 @@ #elif defined(__GNUC__) #include <immintrin.h> #else-#error "Unimplemented!"+#undef IS_X86 /* Unimplemented! */ #endif #endif +#define MAYBE_UNUSED(x) (void)((x))+ #if defined(IS_X86)-static uint64_t xgetbv() {+static uint64_t xgetbv(void) { #if defined(_MSC_VER)   return _xgetbv(0); #else@@ -80,7 +82,7 @@ static #endif     enum cpu_feature-    get_cpu_features() {+    get_cpu_features(void) {    if (g_cpu_features != UNDEFINED) {     return g_cpu_features;@@ -137,6 +139,7 @@                               uint8_t flags) { #if defined(IS_X86)   const enum cpu_feature features = get_cpu_features();+  MAYBE_UNUSED(features); #if !defined(BLAKE3_NO_AVX512)   if (features & AVX512VL) {     blake3_compress_in_place_avx512(cv, block, block_len, counter, flags);@@ -149,7 +152,13 @@     return;   } #endif+#if !defined(BLAKE3_NO_SSE2)+  if (features & SSE2) {+    blake3_compress_in_place_sse2(cv, block, block_len, counter, flags);+    return;+  } #endif+#endif   blake3_compress_in_place_portable(cv, block, block_len, counter, flags); } @@ -159,6 +168,7 @@                          uint8_t out[64]) { #if defined(IS_X86)   const enum cpu_feature features = get_cpu_features();+  MAYBE_UNUSED(features); #if !defined(BLAKE3_NO_AVX512)   if (features & AVX512VL) {     blake3_compress_xof_avx512(cv, block, block_len, counter, flags, out);@@ -171,7 +181,13 @@     return;   } #endif+#if !defined(BLAKE3_NO_SSE2)+  if (features & SSE2) {+    blake3_compress_xof_sse2(cv, block, block_len, counter, flags, out);+    return;+  } #endif+#endif   blake3_compress_xof_portable(cv, block, block_len, counter, flags, out); } @@ -181,6 +197,7 @@                       uint8_t flags_start, uint8_t flags_end, uint8_t *out) { #if defined(IS_X86)   const enum cpu_feature features = get_cpu_features();+  MAYBE_UNUSED(features); #if !defined(BLAKE3_NO_AVX512)   if ((features & (AVX512F|AVX512VL)) == (AVX512F|AVX512VL)) {     blake3_hash_many_avx512(inputs, num_inputs, blocks, key, counter,@@ -205,9 +222,17 @@     return;   } #endif+#if !defined(BLAKE3_NO_SSE2)+  if (features & SSE2) {+    blake3_hash_many_sse2(inputs, num_inputs, blocks, key, counter,+                          increment_counter, flags, flags_start, flags_end,+                          out);+    return;+  } #endif+#endif -#if defined(BLAKE3_USE_NEON)+#if BLAKE3_USE_NEON == 1   blake3_hash_many_neon(inputs, num_inputs, blocks, key, counter,                         increment_counter, flags, flags_start, flags_end, out);   return;@@ -222,6 +247,7 @@ size_t blake3_simd_degree(void) { #if defined(IS_X86)   const enum cpu_feature features = get_cpu_features();+  MAYBE_UNUSED(features); #if !defined(BLAKE3_NO_AVX512)   if ((features & (AVX512F|AVX512VL)) == (AVX512F|AVX512VL)) {     return 16;@@ -237,8 +263,13 @@     return 4;   } #endif+#if !defined(BLAKE3_NO_SSE2)+  if (features & SSE2) {+    return 4;+  } #endif-#if defined(BLAKE3_USE_NEON)+#endif+#if BLAKE3_USE_NEON == 1   return 4; #endif   return 1;

@@ -28,7 +28,7 @@ #define INLINE static inline __attribute__((always_inline)) #endif -#if defined(__x86_64__) || defined(_M_X64)+#if defined(__x86_64__) || defined(_M_X64)  #define IS_X86 #define IS_X86_64 #endif@@ -38,33 +38,33 @@ #define IS_X86_32 #endif +#if defined(__aarch64__) || defined(_M_ARM64)+#define IS_AARCH64+#endif+ #if defined(IS_X86) #if defined(_MSC_VER) #include <intrin.h> #endif-#include <immintrin.h> #endif +#if !defined(BLAKE3_USE_NEON) +  // If BLAKE3_USE_NEON not manually set, autodetect based on AArch64ness+  #if defined(IS_AARCH64)+    #define BLAKE3_USE_NEON 1+  #else+    #define BLAKE3_USE_NEON 0+  #endif+#endif+ #if defined(IS_X86) #define MAX_SIMD_DEGREE 16-#elif defined(BLAKE3_USE_NEON)+#elif BLAKE3_USE_NEON == 1 #define MAX_SIMD_DEGREE 4 #else #define MAX_SIMD_DEGREE 1 #endif -// On GCC and Clang we can enable intrinsics per function, rather than-// requiring their respective -mavx2, -mavx512vl, etc. compiler flags.-#if defined(__GNUC__) || defined(__clang__)-#  define TARGET_AVX2 __attribute__((target("avx2")))-#  define TARGET_AVX512 __attribute__((target("avx512vl,avx512f")))-#  define TARGET_SSE41 __attribute__((target("sse4.1")))-#else-#  define TARGET_AVX2    // On MSVC, use the compiler flag /arch:AVX2-#  define TARGET_AVX512  // On MSVC, use the compiler flag /argc:AVX512-#  define TARGET_SSE41   // On MSVC, this is always enabled.-#endif- // There are some places where we want a static size that's equal to the // MAX_SIMD_DEGREE, but also at least 2. #define MAX_SIMD_DEGREE_OR_2 (MAX_SIMD_DEGREE > 2 ? MAX_SIMD_DEGREE : 2)@@ -87,7 +87,7 @@ /* x is assumed to be nonzero.       */ static unsigned int highest_one(uint64_t x) { #if defined(__GNUC__) || defined(__clang__)-  return 63 ^ __builtin_clzll(x);+  return 63 ^ (unsigned int)__builtin_clzll(x); #elif defined(_MSC_VER) && defined(IS_X86_64)   unsigned long index;   _BitScanReverse64(&index, x);@@ -95,11 +95,11 @@ #elif defined(_MSC_VER) && defined(IS_X86_32)   if(x >> 32) {     unsigned long index;-    _BitScanReverse(&index, x >> 32);+    _BitScanReverse(&index, (unsigned long)(x >> 32));     return 32 + index;   } else {     unsigned long index;-    _BitScanReverse(&index, x);+    _BitScanReverse(&index, (unsigned long)x);     return index;   } #else@@ -117,7 +117,7 @@ // Count the number of 1 bits. INLINE unsigned int popcnt(uint64_t x) { #if defined(__GNUC__) || defined(__clang__)-  return __builtin_popcountll(x);+  return (unsigned int)__builtin_popcountll(x); #else   unsigned int count = 0;   while (x != 0) {@@ -129,7 +129,7 @@ }  // Largest power of two less than or equal to x. As a special case, returns 1-// when x is 0.+// when x is 0.  INLINE uint64_t round_down_to_power_of_2(uint64_t x) {   return 1ULL << highest_one(x | 1); }@@ -158,6 +158,25 @@   key_words[7] = load32(&key[7 * 4]); } +INLINE void store32(void *dst, uint32_t w) {+  uint8_t *p = (uint8_t *)dst;+  p[0] = (uint8_t)(w >> 0);+  p[1] = (uint8_t)(w >> 8);+  p[2] = (uint8_t)(w >> 16);+  p[3] = (uint8_t)(w >> 24);+}++INLINE void store_cv_words(uint8_t bytes_out[32], uint32_t cv_words[8]) {+  store32(&bytes_out[0 * 4], cv_words[0]);+  store32(&bytes_out[1 * 4], cv_words[1]);+  store32(&bytes_out[2 * 4], cv_words[2]);+  store32(&bytes_out[3 * 4], cv_words[3]);+  store32(&bytes_out[4 * 4], cv_words[4]);+  store32(&bytes_out[5 * 4], cv_words[5]);+  store32(&bytes_out[6 * 4], cv_words[6]);+  store32(&bytes_out[7 * 4], cv_words[7]);+}+ void blake3_compress_in_place(uint32_t cv[8],                               const uint8_t block[BLAKE3_BLOCK_LEN],                               uint8_t block_len, uint64_t counter,@@ -194,6 +213,21 @@                                uint8_t flags_end, uint8_t *out);  #if defined(IS_X86)+#if !defined(BLAKE3_NO_SSE2)+void blake3_compress_in_place_sse2(uint32_t cv[8],+                                   const uint8_t block[BLAKE3_BLOCK_LEN],+                                   uint8_t block_len, uint64_t counter,+                                   uint8_t flags);+void blake3_compress_xof_sse2(const uint32_t cv[8],+                              const uint8_t block[BLAKE3_BLOCK_LEN],+                              uint8_t block_len, uint64_t counter,+                              uint8_t flags, uint8_t out[64]);+void blake3_hash_many_sse2(const uint8_t *const *inputs, size_t num_inputs,+                           size_t blocks, const uint32_t key[8],+                           uint64_t counter, bool increment_counter,+                           uint8_t flags, uint8_t flags_start,+                           uint8_t flags_end, uint8_t *out);+#endif #if !defined(BLAKE3_NO_SSE41) void blake3_compress_in_place_sse41(uint32_t cv[8],                                     const uint8_t block[BLAKE3_BLOCK_LEN],@@ -235,12 +269,13 @@ #endif #endif -#if defined(BLAKE3_USE_NEON)+#if BLAKE3_USE_NEON == 1 void blake3_hash_many_neon(const uint8_t *const *inputs, size_t num_inputs,                            size_t blocks, const uint32_t key[8],                            uint64_t counter, bool increment_counter,                            uint8_t flags, uint8_t flags_start,                            uint8_t flags_end, uint8_t *out); #endif+  #endif /* BLAKE3_IMPL_H */

@@ -1,14 +1,6 @@ #include "blake3_impl.h" #include <string.h> -INLINE void store32(void *dst, uint32_t w) {-  uint8_t *p = (uint8_t *)dst;-  p[0] = (uint8_t)(w >> 0);-  p[1] = (uint8_t)(w >> 8);-  p[2] = (uint8_t)(w >> 16);-  p[3] = (uint8_t)(w >> 24);-}- INLINE uint32_t rotr32(uint32_t w, uint32_t c) {   return (w >> c) | (w << (32 - c)); }@@ -147,7 +139,7 @@     blocks -= 1;     block_flags = flags;   }-  memcpy(out, cv, 32);+  store_cv_words(out, cv); }  void blake3_hash_many_portable(const uint8_t *const *inputs, size_t num_inputs,

@@ -0,0 +1,577 @@+#include "blake3_impl.h"++#include <immintrin.h>++#if defined(__clang__)+#pragma clang attribute push (__attribute__((target("sse2"))), apply_to=function)+#elif defined(__GNUC__)+#pragma GCC target("sse2")+#endif++#define DEGREE 4++#define _mm_shuffle_ps2(a, b, c)                                               \+  (_mm_castps_si128(                                                           \+      _mm_shuffle_ps(_mm_castsi128_ps(a), _mm_castsi128_ps(b), (c))))++INLINE __m128i loadu(const uint8_t src[16]) {+  return _mm_loadu_si128((const __m128i *)src);+}++INLINE void storeu(__m128i src, uint8_t dest[16]) {+  _mm_storeu_si128((__m128i *)dest, src);+}++INLINE __m128i addv(__m128i a, __m128i b) { return _mm_add_epi32(a, b); }++// Note that clang-format doesn't like the name "xor" for some reason.+INLINE __m128i xorv(__m128i a, __m128i b) { return _mm_xor_si128(a, b); }++INLINE __m128i set1(uint32_t x) { return _mm_set1_epi32((int32_t)x); }++INLINE __m128i set4(uint32_t a, uint32_t b, uint32_t c, uint32_t d) {+  return _mm_setr_epi32((int32_t)a, (int32_t)b, (int32_t)c, (int32_t)d);+}++INLINE __m128i rot16(__m128i x) {+  return _mm_shufflehi_epi16(_mm_shufflelo_epi16(x, 0xB1), 0xB1);+}++INLINE __m128i rot12(__m128i x) {+  return xorv(_mm_srli_epi32(x, 12), _mm_slli_epi32(x, 32 - 12));+}++INLINE __m128i rot8(__m128i x) {+  return xorv(_mm_srli_epi32(x, 8), _mm_slli_epi32(x, 32 - 8));+}++INLINE __m128i rot7(__m128i x) {+  return xorv(_mm_srli_epi32(x, 7), _mm_slli_epi32(x, 32 - 7));+}++INLINE void g1(__m128i *row0, __m128i *row1, __m128i *row2, __m128i *row3,+               __m128i m) {+  *row0 = addv(addv(*row0, m), *row1);+  *row3 = xorv(*row3, *row0);+  *row3 = rot16(*row3);+  *row2 = addv(*row2, *row3);+  *row1 = xorv(*row1, *row2);+  *row1 = rot12(*row1);+}++INLINE void g2(__m128i *row0, __m128i *row1, __m128i *row2, __m128i *row3,+               __m128i m) {+  *row0 = addv(addv(*row0, m), *row1);+  *row3 = xorv(*row3, *row0);+  *row3 = rot8(*row3);+  *row2 = addv(*row2, *row3);+  *row1 = xorv(*row1, *row2);+  *row1 = rot7(*row1);+}++// Note the optimization here of leaving row1 as the unrotated row, rather than+// row0. All the message loads below are adjusted to compensate for this. See+// discussion at https://github.com/sneves/blake2-avx2/pull/4+INLINE void diagonalize(__m128i *row0, __m128i *row2, __m128i *row3) {+  *row0 = _mm_shuffle_epi32(*row0, _MM_SHUFFLE(2, 1, 0, 3));+  *row3 = _mm_shuffle_epi32(*row3, _MM_SHUFFLE(1, 0, 3, 2));+  *row2 = _mm_shuffle_epi32(*row2, _MM_SHUFFLE(0, 3, 2, 1));+}++INLINE void undiagonalize(__m128i *row0, __m128i *row2, __m128i *row3) {+  *row0 = _mm_shuffle_epi32(*row0, _MM_SHUFFLE(0, 3, 2, 1));+  *row3 = _mm_shuffle_epi32(*row3, _MM_SHUFFLE(1, 0, 3, 2));+  *row2 = _mm_shuffle_epi32(*row2, _MM_SHUFFLE(2, 1, 0, 3));+}++INLINE __m128i blend_epi16(__m128i a, __m128i b, const int16_t imm8) {+  const __m128i bits = _mm_set_epi16(0x80, 0x40, 0x20, 0x10, 0x08, 0x04, 0x02, 0x01);+  __m128i mask = _mm_set1_epi16(imm8);+  mask = _mm_and_si128(mask, bits);+  mask = _mm_cmpeq_epi16(mask, bits);+  return _mm_or_si128(_mm_and_si128(mask, b), _mm_andnot_si128(mask, a));+}++INLINE void compress_pre(__m128i rows[4], const uint32_t cv[8],+                         const uint8_t block[BLAKE3_BLOCK_LEN],+                         uint8_t block_len, uint64_t counter, uint8_t flags) {+  rows[0] = loadu((uint8_t *)&cv[0]);+  rows[1] = loadu((uint8_t *)&cv[4]);+  rows[2] = set4(IV[0], IV[1], IV[2], IV[3]);+  rows[3] = set4(counter_low(counter), counter_high(counter),+                 (uint32_t)block_len, (uint32_t)flags);++  __m128i m0 = loadu(&block[sizeof(__m128i) * 0]);+  __m128i m1 = loadu(&block[sizeof(__m128i) * 1]);+  __m128i m2 = loadu(&block[sizeof(__m128i) * 2]);+  __m128i m3 = loadu(&block[sizeof(__m128i) * 3]);++  __m128i t0, t1, t2, t3, tt;++  // Round 1. The first round permutes the message words from the original+  // input order, into the groups that get mixed in parallel.+  t0 = _mm_shuffle_ps2(m0, m1, _MM_SHUFFLE(2, 0, 2, 0)); //  6  4  2  0+  g1(&rows[0], &rows[1], &rows[2], &rows[3], t0);+  t1 = _mm_shuffle_ps2(m0, m1, _MM_SHUFFLE(3, 1, 3, 1)); //  7  5  3  1+  g2(&rows[0], &rows[1], &rows[2], &rows[3], t1);+  diagonalize(&rows[0], &rows[2], &rows[3]);+  t2 = _mm_shuffle_ps2(m2, m3, _MM_SHUFFLE(2, 0, 2, 0)); // 14 12 10  8+  t2 = _mm_shuffle_epi32(t2, _MM_SHUFFLE(2, 1, 0, 3));   // 12 10  8 14+  g1(&rows[0], &rows[1], &rows[2], &rows[3], t2);+  t3 = _mm_shuffle_ps2(m2, m3, _MM_SHUFFLE(3, 1, 3, 1)); // 15 13 11  9+  t3 = _mm_shuffle_epi32(t3, _MM_SHUFFLE(2, 1, 0, 3));   // 13 11  9 15+  g2(&rows[0], &rows[1], &rows[2], &rows[3], t3);+  undiagonalize(&rows[0], &rows[2], &rows[3]);+  m0 = t0;+  m1 = t1;+  m2 = t2;+  m3 = t3;++  // Round 2. This round and all following rounds apply a fixed permutation+  // to the message words from the round before.+  t0 = _mm_shuffle_ps2(m0, m1, _MM_SHUFFLE(3, 1, 1, 2));+  t0 = _mm_shuffle_epi32(t0, _MM_SHUFFLE(0, 3, 2, 1));+  g1(&rows[0], &rows[1], &rows[2], &rows[3], t0);+  t1 = _mm_shuffle_ps2(m2, m3, _MM_SHUFFLE(3, 3, 2, 2));+  tt = _mm_shuffle_epi32(m0, _MM_SHUFFLE(0, 0, 3, 3));+  t1 = blend_epi16(tt, t1, 0xCC);+  g2(&rows[0], &rows[1], &rows[2], &rows[3], t1);+  diagonalize(&rows[0], &rows[2], &rows[3]);+  t2 = _mm_unpacklo_epi64(m3, m1);+  tt = blend_epi16(t2, m2, 0xC0);+  t2 = _mm_shuffle_epi32(tt, _MM_SHUFFLE(1, 3, 2, 0));+  g1(&rows[0], &rows[1], &rows[2], &rows[3], t2);+  t3 = _mm_unpackhi_epi32(m1, m3);+  tt = _mm_unpacklo_epi32(m2, t3);+  t3 = _mm_shuffle_epi32(tt, _MM_SHUFFLE(0, 1, 3, 2));+  g2(&rows[0], &rows[1], &rows[2], &rows[3], t3);+  undiagonalize(&rows[0], &rows[2], &rows[3]);+  m0 = t0;+  m1 = t1;+  m2 = t2;+  m3 = t3;++  // Round 3+  t0 = _mm_shuffle_ps2(m0, m1, _MM_SHUFFLE(3, 1, 1, 2));+  t0 = _mm_shuffle_epi32(t0, _MM_SHUFFLE(0, 3, 2, 1));+  g1(&rows[0], &rows[1], &rows[2], &rows[3], t0);+  t1 = _mm_shuffle_ps2(m2, m3, _MM_SHUFFLE(3, 3, 2, 2));+  tt = _mm_shuffle_epi32(m0, _MM_SHUFFLE(0, 0, 3, 3));+  t1 = blend_epi16(tt, t1, 0xCC);+  g2(&rows[0], &rows[1], &rows[2], &rows[3], t1);+  diagonalize(&rows[0], &rows[2], &rows[3]);+  t2 = _mm_unpacklo_epi64(m3, m1);+  tt = blend_epi16(t2, m2, 0xC0);+  t2 = _mm_shuffle_epi32(tt, _MM_SHUFFLE(1, 3, 2, 0));+  g1(&rows[0], &rows[1], &rows[2], &rows[3], t2);+  t3 = _mm_unpackhi_epi32(m1, m3);+  tt = _mm_unpacklo_epi32(m2, t3);+  t3 = _mm_shuffle_epi32(tt, _MM_SHUFFLE(0, 1, 3, 2));+  g2(&rows[0], &rows[1], &rows[2], &rows[3], t3);+  undiagonalize(&rows[0], &rows[2], &rows[3]);+  m0 = t0;+  m1 = t1;+  m2 = t2;+  m3 = t3;++  // Round 4+  t0 = _mm_shuffle_ps2(m0, m1, _MM_SHUFFLE(3, 1, 1, 2));+  t0 = _mm_shuffle_epi32(t0, _MM_SHUFFLE(0, 3, 2, 1));+  g1(&rows[0], &rows[1], &rows[2], &rows[3], t0);+  t1 = _mm_shuffle_ps2(m2, m3, _MM_SHUFFLE(3, 3, 2, 2));+  tt = _mm_shuffle_epi32(m0, _MM_SHUFFLE(0, 0, 3, 3));+  t1 = blend_epi16(tt, t1, 0xCC);+  g2(&rows[0], &rows[1], &rows[2], &rows[3], t1);+  diagonalize(&rows[0], &rows[2], &rows[3]);+  t2 = _mm_unpacklo_epi64(m3, m1);+  tt = blend_epi16(t2, m2, 0xC0);+  t2 = _mm_shuffle_epi32(tt, _MM_SHUFFLE(1, 3, 2, 0));+  g1(&rows[0], &rows[1], &rows[2], &rows[3], t2);+  t3 = _mm_unpackhi_epi32(m1, m3);+  tt = _mm_unpacklo_epi32(m2, t3);+  t3 = _mm_shuffle_epi32(tt, _MM_SHUFFLE(0, 1, 3, 2));+  g2(&rows[0], &rows[1], &rows[2], &rows[3], t3);+  undiagonalize(&rows[0], &rows[2], &rows[3]);+  m0 = t0;+  m1 = t1;+  m2 = t2;+  m3 = t3;++  // Round 5+  t0 = _mm_shuffle_ps2(m0, m1, _MM_SHUFFLE(3, 1, 1, 2));+  t0 = _mm_shuffle_epi32(t0, _MM_SHUFFLE(0, 3, 2, 1));+  g1(&rows[0], &rows[1], &rows[2], &rows[3], t0);+  t1 = _mm_shuffle_ps2(m2, m3, _MM_SHUFFLE(3, 3, 2, 2));+  tt = _mm_shuffle_epi32(m0, _MM_SHUFFLE(0, 0, 3, 3));+  t1 = blend_epi16(tt, t1, 0xCC);+  g2(&rows[0], &rows[1], &rows[2], &rows[3], t1);+  diagonalize(&rows[0], &rows[2], &rows[3]);+  t2 = _mm_unpacklo_epi64(m3, m1);+  tt = blend_epi16(t2, m2, 0xC0);+  t2 = _mm_shuffle_epi32(tt, _MM_SHUFFLE(1, 3, 2, 0));+  g1(&rows[0], &rows[1], &rows[2], &rows[3], t2);+  t3 = _mm_unpackhi_epi32(m1, m3);+  tt = _mm_unpacklo_epi32(m2, t3);+  t3 = _mm_shuffle_epi32(tt, _MM_SHUFFLE(0, 1, 3, 2));+  g2(&rows[0], &rows[1], &rows[2], &rows[3], t3);+  undiagonalize(&rows[0], &rows[2], &rows[3]);+  m0 = t0;+  m1 = t1;+  m2 = t2;+  m3 = t3;++  // Round 6+  t0 = _mm_shuffle_ps2(m0, m1, _MM_SHUFFLE(3, 1, 1, 2));+  t0 = _mm_shuffle_epi32(t0, _MM_SHUFFLE(0, 3, 2, 1));+  g1(&rows[0], &rows[1], &rows[2], &rows[3], t0);+  t1 = _mm_shuffle_ps2(m2, m3, _MM_SHUFFLE(3, 3, 2, 2));+  tt = _mm_shuffle_epi32(m0, _MM_SHUFFLE(0, 0, 3, 3));+  t1 = blend_epi16(tt, t1, 0xCC);+  g2(&rows[0], &rows[1], &rows[2], &rows[3], t1);+  diagonalize(&rows[0], &rows[2], &rows[3]);+  t2 = _mm_unpacklo_epi64(m3, m1);+  tt = blend_epi16(t2, m2, 0xC0);+  t2 = _mm_shuffle_epi32(tt, _MM_SHUFFLE(1, 3, 2, 0));+  g1(&rows[0], &rows[1], &rows[2], &rows[3], t2);+  t3 = _mm_unpackhi_epi32(m1, m3);+  tt = _mm_unpacklo_epi32(m2, t3);+  t3 = _mm_shuffle_epi32(tt, _MM_SHUFFLE(0, 1, 3, 2));+  g2(&rows[0], &rows[1], &rows[2], &rows[3], t3);+  undiagonalize(&rows[0], &rows[2], &rows[3]);+  m0 = t0;+  m1 = t1;+  m2 = t2;+  m3 = t3;++  // Round 7+  t0 = _mm_shuffle_ps2(m0, m1, _MM_SHUFFLE(3, 1, 1, 2));+  t0 = _mm_shuffle_epi32(t0, _MM_SHUFFLE(0, 3, 2, 1));+  g1(&rows[0], &rows[1], &rows[2], &rows[3], t0);+  t1 = _mm_shuffle_ps2(m2, m3, _MM_SHUFFLE(3, 3, 2, 2));+  tt = _mm_shuffle_epi32(m0, _MM_SHUFFLE(0, 0, 3, 3));+  t1 = blend_epi16(tt, t1, 0xCC);+  g2(&rows[0], &rows[1], &rows[2], &rows[3], t1);+  diagonalize(&rows[0], &rows[2], &rows[3]);+  t2 = _mm_unpacklo_epi64(m3, m1);+  tt = blend_epi16(t2, m2, 0xC0);+  t2 = _mm_shuffle_epi32(tt, _MM_SHUFFLE(1, 3, 2, 0));+  g1(&rows[0], &rows[1], &rows[2], &rows[3], t2);+  t3 = _mm_unpackhi_epi32(m1, m3);+  tt = _mm_unpacklo_epi32(m2, t3);+  t3 = _mm_shuffle_epi32(tt, _MM_SHUFFLE(0, 1, 3, 2));+  g2(&rows[0], &rows[1], &rows[2], &rows[3], t3);+  undiagonalize(&rows[0], &rows[2], &rows[3]);+}++void blake3_compress_in_place_sse2(uint32_t cv[8],+                                   const uint8_t block[BLAKE3_BLOCK_LEN],+                                   uint8_t block_len, uint64_t counter,+                                   uint8_t flags) {+  __m128i rows[4];+  compress_pre(rows, cv, block, block_len, counter, flags);+  storeu(xorv(rows[0], rows[2]), (uint8_t *)&cv[0]);+  storeu(xorv(rows[1], rows[3]), (uint8_t *)&cv[4]);+}++void blake3_compress_xof_sse2(const uint32_t cv[8],+                              const uint8_t block[BLAKE3_BLOCK_LEN],+                              uint8_t block_len, uint64_t counter,+                              uint8_t flags, uint8_t out[64]) {+  __m128i rows[4];+  compress_pre(rows, cv, block, block_len, counter, flags);+  storeu(xorv(rows[0], rows[2]), &out[0]);+  storeu(xorv(rows[1], rows[3]), &out[16]);+  storeu(xorv(rows[2], loadu((uint8_t *)&cv[0])), &out[32]);+  storeu(xorv(rows[3], loadu((uint8_t *)&cv[4])), &out[48]);+}++INLINE void round_fn(__m128i v[16], __m128i m[16], size_t r) {+  v[0] = addv(v[0], m[(size_t)MSG_SCHEDULE[r][0]]);+  v[1] = addv(v[1], m[(size_t)MSG_SCHEDULE[r][2]]);+  v[2] = addv(v[2], m[(size_t)MSG_SCHEDULE[r][4]]);+  v[3] = addv(v[3], m[(size_t)MSG_SCHEDULE[r][6]]);+  v[0] = addv(v[0], v[4]);+  v[1] = addv(v[1], v[5]);+  v[2] = addv(v[2], v[6]);+  v[3] = addv(v[3], v[7]);+  v[12] = xorv(v[12], v[0]);+  v[13] = xorv(v[13], v[1]);+  v[14] = xorv(v[14], v[2]);+  v[15] = xorv(v[15], v[3]);+  v[12] = rot16(v[12]);+  v[13] = rot16(v[13]);+  v[14] = rot16(v[14]);+  v[15] = rot16(v[15]);+  v[8] = addv(v[8], v[12]);+  v[9] = addv(v[9], v[13]);+  v[10] = addv(v[10], v[14]);+  v[11] = addv(v[11], v[15]);+  v[4] = xorv(v[4], v[8]);+  v[5] = xorv(v[5], v[9]);+  v[6] = xorv(v[6], v[10]);+  v[7] = xorv(v[7], v[11]);+  v[4] = rot12(v[4]);+  v[5] = rot12(v[5]);+  v[6] = rot12(v[6]);+  v[7] = rot12(v[7]);+  v[0] = addv(v[0], m[(size_t)MSG_SCHEDULE[r][1]]);+  v[1] = addv(v[1], m[(size_t)MSG_SCHEDULE[r][3]]);+  v[2] = addv(v[2], m[(size_t)MSG_SCHEDULE[r][5]]);+  v[3] = addv(v[3], m[(size_t)MSG_SCHEDULE[r][7]]);+  v[0] = addv(v[0], v[4]);+  v[1] = addv(v[1], v[5]);+  v[2] = addv(v[2], v[6]);+  v[3] = addv(v[3], v[7]);+  v[12] = xorv(v[12], v[0]);+  v[13] = xorv(v[13], v[1]);+  v[14] = xorv(v[14], v[2]);+  v[15] = xorv(v[15], v[3]);+  v[12] = rot8(v[12]);+  v[13] = rot8(v[13]);+  v[14] = rot8(v[14]);+  v[15] = rot8(v[15]);+  v[8] = addv(v[8], v[12]);+  v[9] = addv(v[9], v[13]);+  v[10] = addv(v[10], v[14]);+  v[11] = addv(v[11], v[15]);+  v[4] = xorv(v[4], v[8]);+  v[5] = xorv(v[5], v[9]);+  v[6] = xorv(v[6], v[10]);+  v[7] = xorv(v[7], v[11]);+  v[4] = rot7(v[4]);+  v[5] = rot7(v[5]);+  v[6] = rot7(v[6]);+  v[7] = rot7(v[7]);++  v[0] = addv(v[0], m[(size_t)MSG_SCHEDULE[r][8]]);+  v[1] = addv(v[1], m[(size_t)MSG_SCHEDULE[r][10]]);+  v[2] = addv(v[2], m[(size_t)MSG_SCHEDULE[r][12]]);+  v[3] = addv(v[3], m[(size_t)MSG_SCHEDULE[r][14]]);+  v[0] = addv(v[0], v[5]);+  v[1] = addv(v[1], v[6]);+  v[2] = addv(v[2], v[7]);+  v[3] = addv(v[3], v[4]);+  v[15] = xorv(v[15], v[0]);+  v[12] = xorv(v[12], v[1]);+  v[13] = xorv(v[13], v[2]);+  v[14] = xorv(v[14], v[3]);+  v[15] = rot16(v[15]);+  v[12] = rot16(v[12]);+  v[13] = rot16(v[13]);+  v[14] = rot16(v[14]);+  v[10] = addv(v[10], v[15]);+  v[11] = addv(v[11], v[12]);+  v[8] = addv(v[8], v[13]);+  v[9] = addv(v[9], v[14]);+  v[5] = xorv(v[5], v[10]);+  v[6] = xorv(v[6], v[11]);+  v[7] = xorv(v[7], v[8]);+  v[4] = xorv(v[4], v[9]);+  v[5] = rot12(v[5]);+  v[6] = rot12(v[6]);+  v[7] = rot12(v[7]);+  v[4] = rot12(v[4]);+  v[0] = addv(v[0], m[(size_t)MSG_SCHEDULE[r][9]]);+  v[1] = addv(v[1], m[(size_t)MSG_SCHEDULE[r][11]]);+  v[2] = addv(v[2], m[(size_t)MSG_SCHEDULE[r][13]]);+  v[3] = addv(v[3], m[(size_t)MSG_SCHEDULE[r][15]]);+  v[0] = addv(v[0], v[5]);+  v[1] = addv(v[1], v[6]);+  v[2] = addv(v[2], v[7]);+  v[3] = addv(v[3], v[4]);+  v[15] = xorv(v[15], v[0]);+  v[12] = xorv(v[12], v[1]);+  v[13] = xorv(v[13], v[2]);+  v[14] = xorv(v[14], v[3]);+  v[15] = rot8(v[15]);+  v[12] = rot8(v[12]);+  v[13] = rot8(v[13]);+  v[14] = rot8(v[14]);+  v[10] = addv(v[10], v[15]);+  v[11] = addv(v[11], v[12]);+  v[8] = addv(v[8], v[13]);+  v[9] = addv(v[9], v[14]);+  v[5] = xorv(v[5], v[10]);+  v[6] = xorv(v[6], v[11]);+  v[7] = xorv(v[7], v[8]);+  v[4] = xorv(v[4], v[9]);+  v[5] = rot7(v[5]);+  v[6] = rot7(v[6]);+  v[7] = rot7(v[7]);+  v[4] = rot7(v[4]);+}++INLINE void transpose_vecs(__m128i vecs[DEGREE]) {+  // Interleave 32-bit lates. The low unpack is lanes 00/11 and the high is+  // 22/33. Note that this doesn't split the vector into two lanes, as the+  // AVX2 counterparts do.+  __m128i ab_01 = _mm_unpacklo_epi32(vecs[0], vecs[1]);+  __m128i ab_23 = _mm_unpackhi_epi32(vecs[0], vecs[1]);+  __m128i cd_01 = _mm_unpacklo_epi32(vecs[2], vecs[3]);+  __m128i cd_23 = _mm_unpackhi_epi32(vecs[2], vecs[3]);++  // Interleave 64-bit lanes.+  __m128i abcd_0 = _mm_unpacklo_epi64(ab_01, cd_01);+  __m128i abcd_1 = _mm_unpackhi_epi64(ab_01, cd_01);+  __m128i abcd_2 = _mm_unpacklo_epi64(ab_23, cd_23);+  __m128i abcd_3 = _mm_unpackhi_epi64(ab_23, cd_23);++  vecs[0] = abcd_0;+  vecs[1] = abcd_1;+  vecs[2] = abcd_2;+  vecs[3] = abcd_3;+}++INLINE void transpose_msg_vecs(const uint8_t *const *inputs,+                               size_t block_offset, __m128i out[16]) {+  out[0] = loadu(&inputs[0][block_offset + 0 * sizeof(__m128i)]);+  out[1] = loadu(&inputs[1][block_offset + 0 * sizeof(__m128i)]);+  out[2] = loadu(&inputs[2][block_offset + 0 * sizeof(__m128i)]);+  out[3] = loadu(&inputs[3][block_offset + 0 * sizeof(__m128i)]);+  out[4] = loadu(&inputs[0][block_offset + 1 * sizeof(__m128i)]);+  out[5] = loadu(&inputs[1][block_offset + 1 * sizeof(__m128i)]);+  out[6] = loadu(&inputs[2][block_offset + 1 * sizeof(__m128i)]);+  out[7] = loadu(&inputs[3][block_offset + 1 * sizeof(__m128i)]);+  out[8] = loadu(&inputs[0][block_offset + 2 * sizeof(__m128i)]);+  out[9] = loadu(&inputs[1][block_offset + 2 * sizeof(__m128i)]);+  out[10] = loadu(&inputs[2][block_offset + 2 * sizeof(__m128i)]);+  out[11] = loadu(&inputs[3][block_offset + 2 * sizeof(__m128i)]);+  out[12] = loadu(&inputs[0][block_offset + 3 * sizeof(__m128i)]);+  out[13] = loadu(&inputs[1][block_offset + 3 * sizeof(__m128i)]);+  out[14] = loadu(&inputs[2][block_offset + 3 * sizeof(__m128i)]);+  out[15] = loadu(&inputs[3][block_offset + 3 * sizeof(__m128i)]);+  for (size_t i = 0; i < 4; ++i) {+    _mm_prefetch((const void *)&inputs[i][block_offset + 256], _MM_HINT_T0);+  }+  transpose_vecs(&out[0]);+  transpose_vecs(&out[4]);+  transpose_vecs(&out[8]);+  transpose_vecs(&out[12]);+}++INLINE void load_counters(uint64_t counter, bool increment_counter,+                          __m128i *out_lo, __m128i *out_hi) {+  const __m128i mask = _mm_set1_epi32(-(int32_t)increment_counter);+  const __m128i add0 = _mm_set_epi32(3, 2, 1, 0);+  const __m128i add1 = _mm_and_si128(mask, add0);+  __m128i l = _mm_add_epi32(_mm_set1_epi32((int32_t)counter), add1);+  __m128i carry = _mm_cmpgt_epi32(_mm_xor_si128(add1, _mm_set1_epi32(0x80000000)),+                                  _mm_xor_si128(   l, _mm_set1_epi32(0x80000000)));+  __m128i h = _mm_sub_epi32(_mm_set1_epi32((int32_t)(counter >> 32)), carry);+  *out_lo = l;+  *out_hi = h;+}++static+void blake3_hash4_sse2(const uint8_t *const *inputs, size_t blocks,+                       const uint32_t key[8], uint64_t counter,+                       bool increment_counter, uint8_t flags,+                       uint8_t flags_start, uint8_t flags_end, uint8_t *out) {+  __m128i h_vecs[8] = {+      set1(key[0]), set1(key[1]), set1(key[2]), set1(key[3]),+      set1(key[4]), set1(key[5]), set1(key[6]), set1(key[7]),+  };+  __m128i counter_low_vec, counter_high_vec;+  load_counters(counter, increment_counter, &counter_low_vec,+                &counter_high_vec);+  uint8_t block_flags = flags | flags_start;++  for (size_t block = 0; block < blocks; block++) {+    if (block + 1 == blocks) {+      block_flags |= flags_end;+    }+    __m128i block_len_vec = set1(BLAKE3_BLOCK_LEN);+    __m128i block_flags_vec = set1(block_flags);+    __m128i msg_vecs[16];+    transpose_msg_vecs(inputs, block * BLAKE3_BLOCK_LEN, msg_vecs);++    __m128i v[16] = {+        h_vecs[0],       h_vecs[1],        h_vecs[2],     h_vecs[3],+        h_vecs[4],       h_vecs[5],        h_vecs[6],     h_vecs[7],+        set1(IV[0]),     set1(IV[1]),      set1(IV[2]),   set1(IV[3]),+        counter_low_vec, counter_high_vec, block_len_vec, block_flags_vec,+    };+    round_fn(v, msg_vecs, 0);+    round_fn(v, msg_vecs, 1);+    round_fn(v, msg_vecs, 2);+    round_fn(v, msg_vecs, 3);+    round_fn(v, msg_vecs, 4);+    round_fn(v, msg_vecs, 5);+    round_fn(v, msg_vecs, 6);+    h_vecs[0] = xorv(v[0], v[8]);+    h_vecs[1] = xorv(v[1], v[9]);+    h_vecs[2] = xorv(v[2], v[10]);+    h_vecs[3] = xorv(v[3], v[11]);+    h_vecs[4] = xorv(v[4], v[12]);+    h_vecs[5] = xorv(v[5], v[13]);+    h_vecs[6] = xorv(v[6], v[14]);+    h_vecs[7] = xorv(v[7], v[15]);++    block_flags = flags;+  }++  transpose_vecs(&h_vecs[0]);+  transpose_vecs(&h_vecs[4]);+  // The first four vecs now contain the first half of each output, and the+  // second four vecs contain the second half of each output.+  storeu(h_vecs[0], &out[0 * sizeof(__m128i)]);+  storeu(h_vecs[4], &out[1 * sizeof(__m128i)]);+  storeu(h_vecs[1], &out[2 * sizeof(__m128i)]);+  storeu(h_vecs[5], &out[3 * sizeof(__m128i)]);+  storeu(h_vecs[2], &out[4 * sizeof(__m128i)]);+  storeu(h_vecs[6], &out[5 * sizeof(__m128i)]);+  storeu(h_vecs[3], &out[6 * sizeof(__m128i)]);+  storeu(h_vecs[7], &out[7 * sizeof(__m128i)]);+}++INLINE void hash_one_sse2(const uint8_t *input, size_t blocks,+                          const uint32_t key[8], uint64_t counter,+                          uint8_t flags, uint8_t flags_start,+                          uint8_t flags_end, uint8_t out[BLAKE3_OUT_LEN]) {+  uint32_t cv[8];+  memcpy(cv, key, BLAKE3_KEY_LEN);+  uint8_t block_flags = flags | flags_start;+  while (blocks > 0) {+    if (blocks == 1) {+      block_flags |= flags_end;+    }+    blake3_compress_in_place_sse2(cv, input, BLAKE3_BLOCK_LEN, counter,+                                  block_flags);+    input = &input[BLAKE3_BLOCK_LEN];+    blocks -= 1;+    block_flags = flags;+  }+  memcpy(out, cv, BLAKE3_OUT_LEN);+}++void blake3_hash_many_sse2(const uint8_t *const *inputs, size_t num_inputs,+                           size_t blocks, const uint32_t key[8],+                           uint64_t counter, bool increment_counter,+                           uint8_t flags, uint8_t flags_start,+                           uint8_t flags_end, uint8_t *out) {+  while (num_inputs >= DEGREE) {+    blake3_hash4_sse2(inputs, blocks, key, counter, increment_counter, flags,+                      flags_start, flags_end, out);+    if (increment_counter) {+      counter += DEGREE;+    }+    inputs += DEGREE;+    num_inputs -= DEGREE;+    out = &out[DEGREE * BLAKE3_OUT_LEN];+  }+  while (num_inputs > 0) {+    hash_one_sse2(inputs[0], blocks, key, counter, flags, flags_start,+                  flags_end, out);+    if (increment_counter) {+      counter += 1;+    }+    inputs += 1;+    num_inputs -= 1;+    out = &out[BLAKE3_OUT_LEN];+  }+}++#if defined(__clang__)+#pragma clang attribute pop+#endif+

@@ -0,0 +1,2291 @@+#if defined(__ELF__) && defined(__linux__)+.section .note.GNU-stack,"",%progbits+#endif++#if defined(__ELF__) && defined(__CET__) && defined(__has_include)+#if __has_include(<cet.h>)+#include <cet.h>+#endif+#endif++#if !defined(_CET_ENDBR)+#define _CET_ENDBR+#endif++.intel_syntax noprefix+.global blake3_hash_many_sse2+.global _blake3_hash_many_sse2+.global blake3_compress_in_place_sse2+.global _blake3_compress_in_place_sse2+.global blake3_compress_xof_sse2+.global _blake3_compress_xof_sse2+#ifdef __APPLE__+.text+#else+.section .text+#endif+        .p2align  6+_blake3_hash_many_sse2:+blake3_hash_many_sse2:+        _CET_ENDBR+        push    r15+        push    r14+        push    r13+        push    r12+        push    rbx+        push    rbp+        mov     rbp, rsp+        sub     rsp, 360+        and     rsp, 0xFFFFFFFFFFFFFFC0+        neg     r9d+        movd    xmm0, r9d+        pshufd  xmm0, xmm0, 0x00+        movdqa  xmmword ptr [rsp+0x130], xmm0+        movdqa  xmm1, xmm0+        pand    xmm1, xmmword ptr [ADD0+rip]+        pand    xmm0, xmmword ptr [ADD1+rip]+        movdqa  xmmword ptr [rsp+0x150], xmm0+        movd    xmm0, r8d+        pshufd  xmm0, xmm0, 0x00+        paddd   xmm0, xmm1+        movdqa  xmmword ptr [rsp+0x110], xmm0+        pxor    xmm0, xmmword ptr [CMP_MSB_MASK+rip]+        pxor    xmm1, xmmword ptr [CMP_MSB_MASK+rip]+        pcmpgtd xmm1, xmm0+        shr     r8, 32+        movd    xmm2, r8d+        pshufd  xmm2, xmm2, 0x00+        psubd   xmm2, xmm1+        movdqa  xmmword ptr [rsp+0x120], xmm2+        mov     rbx, qword ptr [rbp+0x50]+        mov     r15, rdx+        shl     r15, 6+        movzx   r13d, byte ptr [rbp+0x38]+        movzx   r12d, byte ptr [rbp+0x48]+        cmp     rsi, 4+        jc      3f+2:+        movdqu  xmm3, xmmword ptr [rcx]+        pshufd  xmm0, xmm3, 0x00+        pshufd  xmm1, xmm3, 0x55+        pshufd  xmm2, xmm3, 0xAA+        pshufd  xmm3, xmm3, 0xFF+        movdqu  xmm7, xmmword ptr [rcx+0x10]+        pshufd  xmm4, xmm7, 0x00+        pshufd  xmm5, xmm7, 0x55+        pshufd  xmm6, xmm7, 0xAA+        pshufd  xmm7, xmm7, 0xFF+        mov     r8, qword ptr [rdi]+        mov     r9, qword ptr [rdi+0x8]+        mov     r10, qword ptr [rdi+0x10]+        mov     r11, qword ptr [rdi+0x18]+        movzx   eax, byte ptr [rbp+0x40]+        or      eax, r13d+        xor     edx, edx+9:+        mov     r14d, eax+        or      eax, r12d+        add     rdx, 64+        cmp     rdx, r15+        cmovne  eax, r14d+        movdqu  xmm8, xmmword ptr [r8+rdx-0x40]+        movdqu  xmm9, xmmword ptr [r9+rdx-0x40]+        movdqu  xmm10, xmmword ptr [r10+rdx-0x40]+        movdqu  xmm11, xmmword ptr [r11+rdx-0x40]+        movdqa  xmm12, xmm8+        punpckldq xmm8, xmm9+        punpckhdq xmm12, xmm9+        movdqa  xmm14, xmm10+        punpckldq xmm10, xmm11+        punpckhdq xmm14, xmm11+        movdqa  xmm9, xmm8+        punpcklqdq xmm8, xmm10+        punpckhqdq xmm9, xmm10+        movdqa  xmm13, xmm12+        punpcklqdq xmm12, xmm14+        punpckhqdq xmm13, xmm14+        movdqa  xmmword ptr [rsp], xmm8+        movdqa  xmmword ptr [rsp+0x10], xmm9+        movdqa  xmmword ptr [rsp+0x20], xmm12+        movdqa  xmmword ptr [rsp+0x30], xmm13+        movdqu  xmm8, xmmword ptr [r8+rdx-0x30]+        movdqu  xmm9, xmmword ptr [r9+rdx-0x30]+        movdqu  xmm10, xmmword ptr [r10+rdx-0x30]+        movdqu  xmm11, xmmword ptr [r11+rdx-0x30]+        movdqa  xmm12, xmm8+        punpckldq xmm8, xmm9+        punpckhdq xmm12, xmm9+        movdqa  xmm14, xmm10+        punpckldq xmm10, xmm11+        punpckhdq xmm14, xmm11+        movdqa  xmm9, xmm8+        punpcklqdq xmm8, xmm10+        punpckhqdq xmm9, xmm10+        movdqa  xmm13, xmm12+        punpcklqdq xmm12, xmm14+        punpckhqdq xmm13, xmm14+        movdqa  xmmword ptr [rsp+0x40], xmm8+        movdqa  xmmword ptr [rsp+0x50], xmm9+        movdqa  xmmword ptr [rsp+0x60], xmm12+        movdqa  xmmword ptr [rsp+0x70], xmm13+        movdqu  xmm8, xmmword ptr [r8+rdx-0x20]+        movdqu  xmm9, xmmword ptr [r9+rdx-0x20]+        movdqu  xmm10, xmmword ptr [r10+rdx-0x20]+        movdqu  xmm11, xmmword ptr [r11+rdx-0x20]+        movdqa  xmm12, xmm8+        punpckldq xmm8, xmm9+        punpckhdq xmm12, xmm9+        movdqa  xmm14, xmm10+        punpckldq xmm10, xmm11+        punpckhdq xmm14, xmm11+        movdqa  xmm9, xmm8+        punpcklqdq xmm8, xmm10+        punpckhqdq xmm9, xmm10+        movdqa  xmm13, xmm12+        punpcklqdq xmm12, xmm14+        punpckhqdq xmm13, xmm14+        movdqa  xmmword ptr [rsp+0x80], xmm8+        movdqa  xmmword ptr [rsp+0x90], xmm9+        movdqa  xmmword ptr [rsp+0xA0], xmm12+        movdqa  xmmword ptr [rsp+0xB0], xmm13+        movdqu  xmm8, xmmword ptr [r8+rdx-0x10]+        movdqu  xmm9, xmmword ptr [r9+rdx-0x10]+        movdqu  xmm10, xmmword ptr [r10+rdx-0x10]+        movdqu  xmm11, xmmword ptr [r11+rdx-0x10]+        movdqa  xmm12, xmm8+        punpckldq xmm8, xmm9+        punpckhdq xmm12, xmm9+        movdqa  xmm14, xmm10+        punpckldq xmm10, xmm11+        punpckhdq xmm14, xmm11+        movdqa  xmm9, xmm8+        punpcklqdq xmm8, xmm10+        punpckhqdq xmm9, xmm10+        movdqa  xmm13, xmm12+        punpcklqdq xmm12, xmm14+        punpckhqdq xmm13, xmm14+        movdqa  xmmword ptr [rsp+0xC0], xmm8+        movdqa  xmmword ptr [rsp+0xD0], xmm9+        movdqa  xmmword ptr [rsp+0xE0], xmm12+        movdqa  xmmword ptr [rsp+0xF0], xmm13+        movdqa  xmm9, xmmword ptr [BLAKE3_IV_1+rip]+        movdqa  xmm10, xmmword ptr [BLAKE3_IV_2+rip]+        movdqa  xmm11, xmmword ptr [BLAKE3_IV_3+rip]+        movdqa  xmm12, xmmword ptr [rsp+0x110]+        movdqa  xmm13, xmmword ptr [rsp+0x120]+        movdqa  xmm14, xmmword ptr [BLAKE3_BLOCK_LEN+rip]+        movd    xmm15, eax+        pshufd  xmm15, xmm15, 0x00+        prefetcht0 [r8+rdx+0x80]+        prefetcht0 [r9+rdx+0x80]+        prefetcht0 [r10+rdx+0x80]+        prefetcht0 [r11+rdx+0x80]+        paddd   xmm0, xmmword ptr [rsp]+        paddd   xmm1, xmmword ptr [rsp+0x20]+        paddd   xmm2, xmmword ptr [rsp+0x40]+        paddd   xmm3, xmmword ptr [rsp+0x60]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        movdqa  xmm8, xmmword ptr [BLAKE3_IV_0+rip]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x10]+        paddd   xmm1, xmmword ptr [rsp+0x30]+        paddd   xmm2, xmmword ptr [rsp+0x50]+        paddd   xmm3, xmmword ptr [rsp+0x70]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x80]+        paddd   xmm1, xmmword ptr [rsp+0xA0]+        paddd   xmm2, xmmword ptr [rsp+0xC0]+        paddd   xmm3, xmmword ptr [rsp+0xE0]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x90]+        paddd   xmm1, xmmword ptr [rsp+0xB0]+        paddd   xmm2, xmmword ptr [rsp+0xD0]+        paddd   xmm3, xmmword ptr [rsp+0xF0]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x20]+        paddd   xmm1, xmmword ptr [rsp+0x30]+        paddd   xmm2, xmmword ptr [rsp+0x70]+        paddd   xmm3, xmmword ptr [rsp+0x40]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x60]+        paddd   xmm1, xmmword ptr [rsp+0xA0]+        paddd   xmm2, xmmword ptr [rsp]+        paddd   xmm3, xmmword ptr [rsp+0xD0]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x10]+        paddd   xmm1, xmmword ptr [rsp+0xC0]+        paddd   xmm2, xmmword ptr [rsp+0x90]+        paddd   xmm3, xmmword ptr [rsp+0xF0]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xB0]+        paddd   xmm1, xmmword ptr [rsp+0x50]+        paddd   xmm2, xmmword ptr [rsp+0xE0]+        paddd   xmm3, xmmword ptr [rsp+0x80]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x30]+        paddd   xmm1, xmmword ptr [rsp+0xA0]+        paddd   xmm2, xmmword ptr [rsp+0xD0]+        paddd   xmm3, xmmword ptr [rsp+0x70]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x40]+        paddd   xmm1, xmmword ptr [rsp+0xC0]+        paddd   xmm2, xmmword ptr [rsp+0x20]+        paddd   xmm3, xmmword ptr [rsp+0xE0]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x60]+        paddd   xmm1, xmmword ptr [rsp+0x90]+        paddd   xmm2, xmmword ptr [rsp+0xB0]+        paddd   xmm3, xmmword ptr [rsp+0x80]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x50]+        paddd   xmm1, xmmword ptr [rsp]+        paddd   xmm2, xmmword ptr [rsp+0xF0]+        paddd   xmm3, xmmword ptr [rsp+0x10]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xA0]+        paddd   xmm1, xmmword ptr [rsp+0xC0]+        paddd   xmm2, xmmword ptr [rsp+0xE0]+        paddd   xmm3, xmmword ptr [rsp+0xD0]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x70]+        paddd   xmm1, xmmword ptr [rsp+0x90]+        paddd   xmm2, xmmword ptr [rsp+0x30]+        paddd   xmm3, xmmword ptr [rsp+0xF0]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x40]+        paddd   xmm1, xmmword ptr [rsp+0xB0]+        paddd   xmm2, xmmword ptr [rsp+0x50]+        paddd   xmm3, xmmword ptr [rsp+0x10]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp]+        paddd   xmm1, xmmword ptr [rsp+0x20]+        paddd   xmm2, xmmword ptr [rsp+0x80]+        paddd   xmm3, xmmword ptr [rsp+0x60]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xC0]+        paddd   xmm1, xmmword ptr [rsp+0x90]+        paddd   xmm2, xmmword ptr [rsp+0xF0]+        paddd   xmm3, xmmword ptr [rsp+0xE0]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xD0]+        paddd   xmm1, xmmword ptr [rsp+0xB0]+        paddd   xmm2, xmmword ptr [rsp+0xA0]+        paddd   xmm3, xmmword ptr [rsp+0x80]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x70]+        paddd   xmm1, xmmword ptr [rsp+0x50]+        paddd   xmm2, xmmword ptr [rsp]+        paddd   xmm3, xmmword ptr [rsp+0x60]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x20]+        paddd   xmm1, xmmword ptr [rsp+0x30]+        paddd   xmm2, xmmword ptr [rsp+0x10]+        paddd   xmm3, xmmword ptr [rsp+0x40]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x90]+        paddd   xmm1, xmmword ptr [rsp+0xB0]+        paddd   xmm2, xmmword ptr [rsp+0x80]+        paddd   xmm3, xmmword ptr [rsp+0xF0]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xE0]+        paddd   xmm1, xmmword ptr [rsp+0x50]+        paddd   xmm2, xmmword ptr [rsp+0xC0]+        paddd   xmm3, xmmword ptr [rsp+0x10]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xD0]+        paddd   xmm1, xmmword ptr [rsp]+        paddd   xmm2, xmmword ptr [rsp+0x20]+        paddd   xmm3, xmmword ptr [rsp+0x40]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x30]+        paddd   xmm1, xmmword ptr [rsp+0xA0]+        paddd   xmm2, xmmword ptr [rsp+0x60]+        paddd   xmm3, xmmword ptr [rsp+0x70]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xB0]+        paddd   xmm1, xmmword ptr [rsp+0x50]+        paddd   xmm2, xmmword ptr [rsp+0x10]+        paddd   xmm3, xmmword ptr [rsp+0x80]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xF0]+        paddd   xmm1, xmmword ptr [rsp]+        paddd   xmm2, xmmword ptr [rsp+0x90]+        paddd   xmm3, xmmword ptr [rsp+0x60]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xE0]+        paddd   xmm1, xmmword ptr [rsp+0x20]+        paddd   xmm2, xmmword ptr [rsp+0x30]+        paddd   xmm3, xmmword ptr [rsp+0x70]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xA0]+        paddd   xmm1, xmmword ptr [rsp+0xC0]+        paddd   xmm2, xmmword ptr [rsp+0x40]+        paddd   xmm3, xmmword ptr [rsp+0xD0]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        pxor    xmm0, xmm8+        pxor    xmm1, xmm9+        pxor    xmm2, xmm10+        pxor    xmm3, xmm11+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        pxor    xmm4, xmm12+        pxor    xmm5, xmm13+        pxor    xmm6, xmm14+        pxor    xmm7, xmm15+        mov     eax, r13d+        jne     9b+        movdqa  xmm9, xmm0+        punpckldq xmm0, xmm1+        punpckhdq xmm9, xmm1+        movdqa  xmm11, xmm2+        punpckldq xmm2, xmm3+        punpckhdq xmm11, xmm3+        movdqa  xmm1, xmm0+        punpcklqdq xmm0, xmm2+        punpckhqdq xmm1, xmm2+        movdqa  xmm3, xmm9+        punpcklqdq xmm9, xmm11+        punpckhqdq xmm3, xmm11+        movdqu  xmmword ptr [rbx], xmm0+        movdqu  xmmword ptr [rbx+0x20], xmm1+        movdqu  xmmword ptr [rbx+0x40], xmm9+        movdqu  xmmword ptr [rbx+0x60], xmm3+        movdqa  xmm9, xmm4+        punpckldq xmm4, xmm5+        punpckhdq xmm9, xmm5+        movdqa  xmm11, xmm6+        punpckldq xmm6, xmm7+        punpckhdq xmm11, xmm7+        movdqa  xmm5, xmm4+        punpcklqdq xmm4, xmm6+        punpckhqdq xmm5, xmm6+        movdqa  xmm7, xmm9+        punpcklqdq xmm9, xmm11+        punpckhqdq xmm7, xmm11+        movdqu  xmmword ptr [rbx+0x10], xmm4+        movdqu  xmmword ptr [rbx+0x30], xmm5+        movdqu  xmmword ptr [rbx+0x50], xmm9+        movdqu  xmmword ptr [rbx+0x70], xmm7+        movdqa  xmm1, xmmword ptr [rsp+0x110]+        movdqa  xmm0, xmm1+        paddd   xmm1, xmmword ptr [rsp+0x150]+        movdqa  xmmword ptr [rsp+0x110], xmm1+        pxor    xmm0, xmmword ptr [CMP_MSB_MASK+rip]+        pxor    xmm1, xmmword ptr [CMP_MSB_MASK+rip]+        pcmpgtd xmm0, xmm1+        movdqa  xmm1, xmmword ptr [rsp+0x120]+        psubd   xmm1, xmm0+        movdqa  xmmword ptr [rsp+0x120], xmm1+        add     rbx, 128+        add     rdi, 32+        sub     rsi, 4+        cmp     rsi, 4+        jnc     2b+        test    rsi, rsi+        jnz     3f+4:+        mov     rsp, rbp+        pop     rbp+        pop     rbx+        pop     r12+        pop     r13+        pop     r14+        pop     r15+        ret+.p2align 5+3:+        test    esi, 0x2+        je      3f+        movups  xmm0, xmmword ptr [rcx]+        movups  xmm1, xmmword ptr [rcx+0x10]+        movaps  xmm8, xmm0+        movaps  xmm9, xmm1+        movd    xmm13, dword ptr [rsp+0x110]+        movd    xmm14, dword ptr [rsp+0x120]+        punpckldq xmm13, xmm14+        movaps  xmmword ptr [rsp], xmm13+        movd    xmm14, dword ptr [rsp+0x114]+        movd    xmm13, dword ptr [rsp+0x124]+        punpckldq xmm14, xmm13+        movaps  xmmword ptr [rsp+0x10], xmm14+        mov     r8, qword ptr [rdi]+        mov     r9, qword ptr [rdi+0x8]+        movzx   eax, byte ptr [rbp+0x40]+        or      eax, r13d+        xor     edx, edx+2:+        mov     r14d, eax+        or      eax, r12d+        add     rdx, 64+        cmp     rdx, r15+        cmovne  eax, r14d+        movaps  xmm2, xmmword ptr [BLAKE3_IV+rip]+        movaps  xmm10, xmm2+        movups  xmm4, xmmword ptr [r8+rdx-0x40]+        movups  xmm5, xmmword ptr [r8+rdx-0x30]+        movaps  xmm3, xmm4+        shufps  xmm4, xmm5, 136+        shufps  xmm3, xmm5, 221+        movaps  xmm5, xmm3+        movups  xmm6, xmmword ptr [r8+rdx-0x20]+        movups  xmm7, xmmword ptr [r8+rdx-0x10]+        movaps  xmm3, xmm6+        shufps  xmm6, xmm7, 136+        pshufd  xmm6, xmm6, 0x93+        shufps  xmm3, xmm7, 221+        pshufd  xmm7, xmm3, 0x93+        movups  xmm12, xmmword ptr [r9+rdx-0x40]+        movups  xmm13, xmmword ptr [r9+rdx-0x30]+        movaps  xmm11, xmm12+        shufps  xmm12, xmm13, 136+        shufps  xmm11, xmm13, 221+        movaps  xmm13, xmm11+        movups  xmm14, xmmword ptr [r9+rdx-0x20]+        movups  xmm15, xmmword ptr [r9+rdx-0x10]+        movaps  xmm11, xmm14+        shufps  xmm14, xmm15, 136+        pshufd  xmm14, xmm14, 0x93+        shufps  xmm11, xmm15, 221+        pshufd  xmm15, xmm11, 0x93+        shl     rax, 0x20+        or      rax, 0x40+        movq    xmm3, rax+        movdqa  xmmword ptr [rsp+0x20], xmm3+        movaps  xmm3, xmmword ptr [rsp]+        movaps  xmm11, xmmword ptr [rsp+0x10]+        punpcklqdq xmm3, xmmword ptr [rsp+0x20]+        punpcklqdq xmm11, xmmword ptr [rsp+0x20]+        mov     al, 7+9:+        paddd   xmm0, xmm4+        paddd   xmm8, xmm12+        movaps  xmmword ptr [rsp+0x20], xmm4+        movaps  xmmword ptr [rsp+0x30], xmm12+        paddd   xmm0, xmm1+        paddd   xmm8, xmm9+        pxor    xmm3, xmm0+        pxor    xmm11, xmm8+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        pshuflw xmm11, xmm11, 0xB1+        pshufhw xmm11, xmm11, 0xB1+        paddd   xmm2, xmm3+        paddd   xmm10, xmm11+        pxor    xmm1, xmm2+        pxor    xmm9, xmm10+        movdqa  xmm4, xmm1+        pslld   xmm1, 20+        psrld   xmm4, 12+        por     xmm1, xmm4+        movdqa  xmm4, xmm9+        pslld   xmm9, 20+        psrld   xmm4, 12+        por     xmm9, xmm4+        paddd   xmm0, xmm5+        paddd   xmm8, xmm13+        movaps  xmmword ptr [rsp+0x40], xmm5+        movaps  xmmword ptr [rsp+0x50], xmm13+        paddd   xmm0, xmm1+        paddd   xmm8, xmm9+        pxor    xmm3, xmm0+        pxor    xmm11, xmm8+        movdqa  xmm13, xmm3+        psrld   xmm3, 8+        pslld   xmm13, 24+        pxor    xmm3, xmm13+        movdqa  xmm13, xmm11+        psrld   xmm11, 8+        pslld   xmm13, 24+        pxor    xmm11, xmm13+        paddd   xmm2, xmm3+        paddd   xmm10, xmm11+        pxor    xmm1, xmm2+        pxor    xmm9, xmm10+        movdqa  xmm4, xmm1+        pslld   xmm1, 25+        psrld   xmm4, 7+        por     xmm1, xmm4+        movdqa  xmm4, xmm9+        pslld   xmm9, 25+        psrld   xmm4, 7+        por     xmm9, xmm4+        pshufd  xmm0, xmm0, 0x93+        pshufd  xmm8, xmm8, 0x93+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm11, xmm11, 0x4E+        pshufd  xmm2, xmm2, 0x39+        pshufd  xmm10, xmm10, 0x39+        paddd   xmm0, xmm6+        paddd   xmm8, xmm14+        paddd   xmm0, xmm1+        paddd   xmm8, xmm9+        pxor    xmm3, xmm0+        pxor    xmm11, xmm8+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        pshuflw xmm11, xmm11, 0xB1+        pshufhw xmm11, xmm11, 0xB1+        paddd   xmm2, xmm3+        paddd   xmm10, xmm11+        pxor    xmm1, xmm2+        pxor    xmm9, xmm10+        movdqa  xmm4, xmm1+        pslld   xmm1, 20+        psrld   xmm4, 12+        por     xmm1, xmm4+        movdqa  xmm4, xmm9+        pslld   xmm9, 20+        psrld   xmm4, 12+        por     xmm9, xmm4+        paddd   xmm0, xmm7+        paddd   xmm8, xmm15+        paddd   xmm0, xmm1+        paddd   xmm8, xmm9+        pxor    xmm3, xmm0+        pxor    xmm11, xmm8+        movdqa  xmm13, xmm3+        psrld   xmm3, 8+        pslld   xmm13, 24+        pxor    xmm3, xmm13+        movdqa  xmm13, xmm11+        psrld   xmm11, 8+        pslld   xmm13, 24+        pxor    xmm11, xmm13+        paddd   xmm2, xmm3+        paddd   xmm10, xmm11+        pxor    xmm1, xmm2+        pxor    xmm9, xmm10+        movdqa  xmm4, xmm1+        pslld   xmm1, 25+        psrld   xmm4, 7+        por     xmm1, xmm4+        movdqa  xmm4, xmm9+        pslld   xmm9, 25+        psrld   xmm4, 7+        por     xmm9, xmm4+        pshufd  xmm0, xmm0, 0x39+        pshufd  xmm8, xmm8, 0x39+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm11, xmm11, 0x4E+        pshufd  xmm2, xmm2, 0x93+        pshufd  xmm10, xmm10, 0x93+        dec     al+        je      9f+        movdqa  xmm12, xmmword ptr [rsp+0x20]+        movdqa  xmm5, xmmword ptr [rsp+0x40]+        pshufd  xmm13, xmm12, 0x0F+        shufps  xmm12, xmm5, 214+        pshufd  xmm4, xmm12, 0x39+        movdqa  xmm12, xmm6+        shufps  xmm12, xmm7, 250+        pand    xmm13, xmmword ptr [PBLENDW_0x33_MASK+rip]+        pand    xmm12, xmmword ptr [PBLENDW_0xCC_MASK+rip]+        por     xmm13, xmm12+        movdqa  xmmword ptr [rsp+0x20], xmm13+        movdqa  xmm12, xmm7+        punpcklqdq xmm12, xmm5+        movdqa  xmm13, xmm6+        pand    xmm12, xmmword ptr [PBLENDW_0x3F_MASK+rip]+        pand    xmm13, xmmword ptr [PBLENDW_0xC0_MASK+rip]+        por     xmm12, xmm13+        pshufd  xmm12, xmm12, 0x78+        punpckhdq xmm5, xmm7+        punpckldq xmm6, xmm5+        pshufd  xmm7, xmm6, 0x1E+        movdqa  xmmword ptr [rsp+0x40], xmm12+        movdqa  xmm5, xmmword ptr [rsp+0x30]+        movdqa  xmm13, xmmword ptr [rsp+0x50]+        pshufd  xmm6, xmm5, 0x0F+        shufps  xmm5, xmm13, 214+        pshufd  xmm12, xmm5, 0x39+        movdqa  xmm5, xmm14+        shufps  xmm5, xmm15, 250+        pand    xmm6, xmmword ptr [PBLENDW_0x33_MASK+rip]+        pand    xmm5, xmmword ptr [PBLENDW_0xCC_MASK+rip]+        por     xmm6, xmm5+        movdqa  xmm5, xmm15+        punpcklqdq xmm5, xmm13+        movdqa  xmmword ptr [rsp+0x30], xmm2+        movdqa  xmm2, xmm14+        pand    xmm5, xmmword ptr [PBLENDW_0x3F_MASK+rip]+        pand    xmm2, xmmword ptr [PBLENDW_0xC0_MASK+rip]+        por     xmm5, xmm2+        movdqa  xmm2, xmmword ptr [rsp+0x30]+        pshufd  xmm5, xmm5, 0x78+        punpckhdq xmm13, xmm15+        punpckldq xmm14, xmm13+        pshufd  xmm15, xmm14, 0x1E+        movdqa  xmm13, xmm6+        movdqa  xmm14, xmm5+        movdqa  xmm5, xmmword ptr [rsp+0x20]+        movdqa  xmm6, xmmword ptr [rsp+0x40]+        jmp     9b+9:+        pxor    xmm0, xmm2+        pxor    xmm1, xmm3+        pxor    xmm8, xmm10+        pxor    xmm9, xmm11+        mov     eax, r13d+        cmp     rdx, r15+        jne     2b+        movups  xmmword ptr [rbx], xmm0+        movups  xmmword ptr [rbx+0x10], xmm1+        movups  xmmword ptr [rbx+0x20], xmm8+        movups  xmmword ptr [rbx+0x30], xmm9+        mov     eax, dword ptr [rsp+0x130]+        neg     eax+        mov    r10d, dword ptr [rsp+0x110+8*rax]+        mov    r11d, dword ptr [rsp+0x120+8*rax]+        mov dword ptr [rsp+0x110], r10d+        mov dword ptr [rsp+0x120], r11d+        add     rdi, 16+        add     rbx, 64+        sub     rsi, 2+3:+        test    esi, 0x1+        je      4b+        movups  xmm0, xmmword ptr [rcx]+        movups  xmm1, xmmword ptr [rcx+0x10]+        movd    xmm13, dword ptr [rsp+0x110]+        movd    xmm14, dword ptr [rsp+0x120]+        punpckldq xmm13, xmm14+        mov     r8, qword ptr [rdi]+        movzx   eax, byte ptr [rbp+0x40]+        or      eax, r13d+        xor     edx, edx+2:+        mov     r14d, eax+        or      eax, r12d+        add     rdx, 64+        cmp     rdx, r15+        cmovne  eax, r14d+        movaps  xmm2, xmmword ptr [BLAKE3_IV+rip]+        shl     rax, 32+        or      rax, 64+        movq    xmm12, rax+        movdqa  xmm3, xmm13+        punpcklqdq xmm3, xmm12+        movups  xmm4, xmmword ptr [r8+rdx-0x40]+        movups  xmm5, xmmword ptr [r8+rdx-0x30]+        movaps  xmm8, xmm4+        shufps  xmm4, xmm5, 136+        shufps  xmm8, xmm5, 221+        movaps  xmm5, xmm8+        movups  xmm6, xmmword ptr [r8+rdx-0x20]+        movups  xmm7, xmmword ptr [r8+rdx-0x10]+        movaps  xmm8, xmm6+        shufps  xmm6, xmm7, 136+        pshufd  xmm6, xmm6, 0x93+        shufps  xmm8, xmm7, 221+        pshufd  xmm7, xmm8, 0x93+        mov     al, 7+9:+        paddd   xmm0, xmm4+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 20+        psrld   xmm11, 12+        por     xmm1, xmm11+        paddd   xmm0, xmm5+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        movdqa  xmm14, xmm3+        psrld   xmm3, 8+        pslld   xmm14, 24+        pxor    xmm3, xmm14+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 25+        psrld   xmm11, 7+        por     xmm1, xmm11+        pshufd  xmm0, xmm0, 0x93+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm2, xmm2, 0x39+        paddd   xmm0, xmm6+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 20+        psrld   xmm11, 12+        por     xmm1, xmm11+        paddd   xmm0, xmm7+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        movdqa  xmm14, xmm3+        psrld   xmm3, 8+        pslld   xmm14, 24+        pxor    xmm3, xmm14+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 25+        psrld   xmm11, 7+        por     xmm1, xmm11+        pshufd  xmm0, xmm0, 0x39+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm2, xmm2, 0x93+        dec     al+        jz      9f+        movdqa  xmm8, xmm4+        shufps  xmm8, xmm5, 214+        pshufd  xmm9, xmm4, 0x0F+        pshufd  xmm4, xmm8, 0x39+        movdqa  xmm8, xmm6+        shufps  xmm8, xmm7, 250+        pand    xmm9, xmmword ptr [PBLENDW_0x33_MASK+rip]+        pand    xmm8, xmmword ptr [PBLENDW_0xCC_MASK+rip]+        por     xmm9, xmm8+        movdqa  xmm8, xmm7+        punpcklqdq xmm8, xmm5+        movdqa  xmm10, xmm6+        pand    xmm8, xmmword ptr [PBLENDW_0x3F_MASK+rip]+        pand    xmm10, xmmword ptr [PBLENDW_0xC0_MASK+rip]+        por     xmm8, xmm10+        pshufd  xmm8, xmm8, 0x78+        punpckhdq xmm5, xmm7+        punpckldq xmm6, xmm5+        pshufd  xmm7, xmm6, 0x1E+        movdqa  xmm5, xmm9+        movdqa  xmm6, xmm8+        jmp     9b+9:+        pxor    xmm0, xmm2+        pxor    xmm1, xmm3+        mov     eax, r13d+        cmp     rdx, r15+        jne     2b+        movups  xmmword ptr [rbx], xmm0+        movups  xmmword ptr [rbx+0x10], xmm1+        jmp     4b++.p2align 6+blake3_compress_in_place_sse2:+_blake3_compress_in_place_sse2:+        _CET_ENDBR+        movups  xmm0, xmmword ptr [rdi]+        movups  xmm1, xmmword ptr [rdi+0x10]+        movaps  xmm2, xmmword ptr [BLAKE3_IV+rip]+        shl     r8, 32+        add     rdx, r8+        movq    xmm3, rcx+        movq    xmm4, rdx+        punpcklqdq xmm3, xmm4+        movups  xmm4, xmmword ptr [rsi]+        movups  xmm5, xmmword ptr [rsi+0x10]+        movaps  xmm8, xmm4+        shufps  xmm4, xmm5, 136+        shufps  xmm8, xmm5, 221+        movaps  xmm5, xmm8+        movups  xmm6, xmmword ptr [rsi+0x20]+        movups  xmm7, xmmword ptr [rsi+0x30]+        movaps  xmm8, xmm6+        shufps  xmm6, xmm7, 136+        pshufd  xmm6, xmm6, 0x93+        shufps  xmm8, xmm7, 221+        pshufd  xmm7, xmm8, 0x93+        mov     al, 7+9:+        paddd   xmm0, xmm4+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 20+        psrld   xmm11, 12+        por     xmm1, xmm11+        paddd   xmm0, xmm5+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        movdqa  xmm14, xmm3+        psrld   xmm3, 8+        pslld   xmm14, 24+        pxor    xmm3, xmm14+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 25+        psrld   xmm11, 7+        por     xmm1, xmm11+        pshufd  xmm0, xmm0, 0x93+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm2, xmm2, 0x39+        paddd   xmm0, xmm6+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 20+        psrld   xmm11, 12+        por     xmm1, xmm11+        paddd   xmm0, xmm7+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        movdqa  xmm14, xmm3+        psrld   xmm3, 8+        pslld   xmm14, 24+        pxor    xmm3, xmm14+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 25+        psrld   xmm11, 7+        por     xmm1, xmm11+        pshufd  xmm0, xmm0, 0x39+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm2, xmm2, 0x93+        dec     al+        jz      9f+        movdqa  xmm8, xmm4+        shufps  xmm8, xmm5, 214+        pshufd  xmm9, xmm4, 0x0F+        pshufd  xmm4, xmm8, 0x39+        movdqa  xmm8, xmm6+        shufps  xmm8, xmm7, 250+        pand    xmm9, xmmword ptr [PBLENDW_0x33_MASK+rip]+        pand    xmm8, xmmword ptr [PBLENDW_0xCC_MASK+rip]+        por     xmm9, xmm8+        movdqa  xmm8, xmm7+        punpcklqdq xmm8, xmm5+        movdqa  xmm10, xmm6+        pand    xmm8, xmmword ptr [PBLENDW_0x3F_MASK+rip]+        pand    xmm10, xmmword ptr [PBLENDW_0xC0_MASK+rip]+        por     xmm8, xmm10+        pshufd  xmm8, xmm8, 0x78+        punpckhdq xmm5, xmm7+        punpckldq xmm6, xmm5+        pshufd  xmm7, xmm6, 0x1E+        movdqa  xmm5, xmm9+        movdqa  xmm6, xmm8+        jmp     9b+9:+        pxor    xmm0, xmm2+        pxor    xmm1, xmm3+        movups  xmmword ptr [rdi], xmm0+        movups  xmmword ptr [rdi+0x10], xmm1+        ret++.p2align 6+blake3_compress_xof_sse2:+_blake3_compress_xof_sse2:+        _CET_ENDBR+        movups  xmm0, xmmword ptr [rdi]+        movups  xmm1, xmmword ptr [rdi+0x10]+        movaps  xmm2, xmmword ptr [BLAKE3_IV+rip]+        movzx   eax, r8b+        movzx   edx, dl+        shl     rax, 32+        add     rdx, rax+        movq    xmm3, rcx+        movq    xmm4, rdx+        punpcklqdq xmm3, xmm4+        movups  xmm4, xmmword ptr [rsi]+        movups  xmm5, xmmword ptr [rsi+0x10]+        movaps  xmm8, xmm4+        shufps  xmm4, xmm5, 136+        shufps  xmm8, xmm5, 221+        movaps  xmm5, xmm8+        movups  xmm6, xmmword ptr [rsi+0x20]+        movups  xmm7, xmmword ptr [rsi+0x30]+        movaps  xmm8, xmm6+        shufps  xmm6, xmm7, 136+        pshufd  xmm6, xmm6, 0x93+        shufps  xmm8, xmm7, 221+        pshufd  xmm7, xmm8, 0x93+        mov     al, 7+9:+        paddd   xmm0, xmm4+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 20+        psrld   xmm11, 12+        por     xmm1, xmm11+        paddd   xmm0, xmm5+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        movdqa  xmm14, xmm3+        psrld   xmm3, 8+        pslld   xmm14, 24+        pxor    xmm3, xmm14+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 25+        psrld   xmm11, 7+        por     xmm1, xmm11+        pshufd  xmm0, xmm0, 0x93+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm2, xmm2, 0x39+        paddd   xmm0, xmm6+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 20+        psrld   xmm11, 12+        por     xmm1, xmm11+        paddd   xmm0, xmm7+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        movdqa  xmm14, xmm3+        psrld   xmm3, 8+        pslld   xmm14, 24+        pxor    xmm3, xmm14+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 25+        psrld   xmm11, 7+        por     xmm1, xmm11+        pshufd  xmm0, xmm0, 0x39+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm2, xmm2, 0x93+        dec     al+        jz      9f+        movdqa  xmm8, xmm4+        shufps  xmm8, xmm5, 214+        pshufd  xmm9, xmm4, 0x0F+        pshufd  xmm4, xmm8, 0x39+        movdqa  xmm8, xmm6+        shufps  xmm8, xmm7, 250+        pand    xmm9, xmmword ptr [PBLENDW_0x33_MASK+rip]+        pand    xmm8, xmmword ptr [PBLENDW_0xCC_MASK+rip]+        por     xmm9, xmm8+        movdqa  xmm8, xmm7+        punpcklqdq xmm8, xmm5+        movdqa  xmm10, xmm6+        pand    xmm8, xmmword ptr [PBLENDW_0x3F_MASK+rip]+        pand    xmm10, xmmword ptr [PBLENDW_0xC0_MASK+rip]+        por     xmm8, xmm10+        pshufd  xmm8, xmm8, 0x78+        punpckhdq xmm5, xmm7+        punpckldq xmm6, xmm5+        pshufd  xmm7, xmm6, 0x1E+        movdqa  xmm5, xmm9+        movdqa  xmm6, xmm8+        jmp     9b+9:+        movdqu  xmm4, xmmword ptr [rdi]+        movdqu  xmm5, xmmword ptr [rdi+0x10]+        pxor    xmm0, xmm2+        pxor    xmm1, xmm3+        pxor    xmm2, xmm4+        pxor    xmm3, xmm5+        movups  xmmword ptr [r9], xmm0+        movups  xmmword ptr [r9+0x10], xmm1+        movups  xmmword ptr [r9+0x20], xmm2+        movups  xmmword ptr [r9+0x30], xmm3+        ret+++#ifdef __APPLE__+.static_data+#else+.section .rodata+#endif+.p2align  6+BLAKE3_IV:+        .long  0x6A09E667, 0xBB67AE85+        .long  0x3C6EF372, 0xA54FF53A+ADD0:	+        .long  0, 1, 2, 3+ADD1:+	.long  4, 4, 4, 4+BLAKE3_IV_0:+	.long  0x6A09E667, 0x6A09E667, 0x6A09E667, 0x6A09E667+BLAKE3_IV_1:+	.long  0xBB67AE85, 0xBB67AE85, 0xBB67AE85, 0xBB67AE85+BLAKE3_IV_2:+	.long  0x3C6EF372, 0x3C6EF372, 0x3C6EF372, 0x3C6EF372+BLAKE3_IV_3:+	.long  0xA54FF53A, 0xA54FF53A, 0xA54FF53A, 0xA54FF53A+BLAKE3_BLOCK_LEN:+	.long  64, 64, 64, 64+CMP_MSB_MASK:+	.long  0x80000000, 0x80000000, 0x80000000, 0x80000000+PBLENDW_0x33_MASK:+	.long  0xFFFFFFFF, 0x00000000, 0xFFFFFFFF, 0x00000000+PBLENDW_0xCC_MASK:+	.long  0x00000000, 0xFFFFFFFF, 0x00000000, 0xFFFFFFFF+PBLENDW_0x3F_MASK:+	.long  0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0x00000000+PBLENDW_0xC0_MASK:+	.long  0x00000000, 0x00000000, 0x00000000, 0xFFFFFFFF

@@ -0,0 +1,2332 @@+.intel_syntax noprefix+.global blake3_hash_many_sse2+.global _blake3_hash_many_sse2+.global blake3_compress_in_place_sse2+.global _blake3_compress_in_place_sse2+.global blake3_compress_xof_sse2+.global _blake3_compress_xof_sse2+.section .text+        .p2align  6+_blake3_hash_many_sse2:+blake3_hash_many_sse2:+        push    r15+        push    r14+        push    r13+        push    r12+        push    rsi+        push    rdi+        push    rbx+        push    rbp+        mov     rbp, rsp+        sub     rsp, 528+        and     rsp, 0xFFFFFFFFFFFFFFC0+        movdqa  xmmword ptr [rsp+0x170], xmm6+        movdqa  xmmword ptr [rsp+0x180], xmm7+        movdqa  xmmword ptr [rsp+0x190], xmm8+        movdqa  xmmword ptr [rsp+0x1A0], xmm9+        movdqa  xmmword ptr [rsp+0x1B0], xmm10+        movdqa  xmmword ptr [rsp+0x1C0], xmm11+        movdqa  xmmword ptr [rsp+0x1D0], xmm12+        movdqa  xmmword ptr [rsp+0x1E0], xmm13+        movdqa  xmmword ptr [rsp+0x1F0], xmm14+        movdqa  xmmword ptr [rsp+0x200], xmm15+        mov     rdi, rcx+        mov     rsi, rdx+        mov     rdx, r8+        mov     rcx, r9+        mov     r8, qword ptr [rbp+0x68]+        movzx   r9, byte ptr [rbp+0x70]+        neg     r9d+        movd    xmm0, r9d+        pshufd  xmm0, xmm0, 0x00+        movdqa  xmmword ptr [rsp+0x130], xmm0+        movdqa  xmm1, xmm0+        pand    xmm1, xmmword ptr [ADD0+rip]+        pand    xmm0, xmmword ptr [ADD1+rip]+        movdqa  xmmword ptr [rsp+0x150], xmm0+        movd    xmm0, r8d+        pshufd  xmm0, xmm0, 0x00+        paddd   xmm0, xmm1+        movdqa  xmmword ptr [rsp+0x110], xmm0+        pxor    xmm0, xmmword ptr [CMP_MSB_MASK+rip]+        pxor    xmm1, xmmword ptr [CMP_MSB_MASK+rip]+        pcmpgtd xmm1, xmm0+        shr     r8, 32+        movd    xmm2, r8d+        pshufd  xmm2, xmm2, 0x00+        psubd   xmm2, xmm1+        movdqa  xmmword ptr [rsp+0x120], xmm2+        mov     rbx, qword ptr [rbp+0x90]+        mov     r15, rdx+        shl     r15, 6+        movzx   r13d, byte ptr [rbp+0x78]+        movzx   r12d, byte ptr [rbp+0x88]+        cmp     rsi, 4+        jc      3f+2:+        movdqu  xmm3, xmmword ptr [rcx]+        pshufd  xmm0, xmm3, 0x00+        pshufd  xmm1, xmm3, 0x55+        pshufd  xmm2, xmm3, 0xAA+        pshufd  xmm3, xmm3, 0xFF+        movdqu  xmm7, xmmword ptr [rcx+0x10]+        pshufd  xmm4, xmm7, 0x00+        pshufd  xmm5, xmm7, 0x55+        pshufd  xmm6, xmm7, 0xAA+        pshufd  xmm7, xmm7, 0xFF+        mov     r8, qword ptr [rdi]+        mov     r9, qword ptr [rdi+0x8]+        mov     r10, qword ptr [rdi+0x10]+        mov     r11, qword ptr [rdi+0x18]+        movzx   eax, byte ptr [rbp+0x80]+        or      eax, r13d+        xor     edx, edx+9:+        mov     r14d, eax+        or      eax, r12d+        add     rdx, 64+        cmp     rdx, r15+        cmovne  eax, r14d+        movdqu  xmm8, xmmword ptr [r8+rdx-0x40]+        movdqu  xmm9, xmmword ptr [r9+rdx-0x40]+        movdqu  xmm10, xmmword ptr [r10+rdx-0x40]+        movdqu  xmm11, xmmword ptr [r11+rdx-0x40]+        movdqa  xmm12, xmm8+        punpckldq xmm8, xmm9+        punpckhdq xmm12, xmm9+        movdqa  xmm14, xmm10+        punpckldq xmm10, xmm11+        punpckhdq xmm14, xmm11+        movdqa  xmm9, xmm8+        punpcklqdq xmm8, xmm10+        punpckhqdq xmm9, xmm10+        movdqa  xmm13, xmm12+        punpcklqdq xmm12, xmm14+        punpckhqdq xmm13, xmm14+        movdqa  xmmword ptr [rsp], xmm8+        movdqa  xmmword ptr [rsp+0x10], xmm9+        movdqa  xmmword ptr [rsp+0x20], xmm12+        movdqa  xmmword ptr [rsp+0x30], xmm13+        movdqu  xmm8, xmmword ptr [r8+rdx-0x30]+        movdqu  xmm9, xmmword ptr [r9+rdx-0x30]+        movdqu  xmm10, xmmword ptr [r10+rdx-0x30]+        movdqu  xmm11, xmmword ptr [r11+rdx-0x30]+        movdqa  xmm12, xmm8+        punpckldq xmm8, xmm9+        punpckhdq xmm12, xmm9+        movdqa  xmm14, xmm10+        punpckldq xmm10, xmm11+        punpckhdq xmm14, xmm11+        movdqa  xmm9, xmm8+        punpcklqdq xmm8, xmm10+        punpckhqdq xmm9, xmm10+        movdqa  xmm13, xmm12+        punpcklqdq xmm12, xmm14+        punpckhqdq xmm13, xmm14+        movdqa  xmmword ptr [rsp+0x40], xmm8+        movdqa  xmmword ptr [rsp+0x50], xmm9+        movdqa  xmmword ptr [rsp+0x60], xmm12+        movdqa  xmmword ptr [rsp+0x70], xmm13+        movdqu  xmm8, xmmword ptr [r8+rdx-0x20]+        movdqu  xmm9, xmmword ptr [r9+rdx-0x20]+        movdqu  xmm10, xmmword ptr [r10+rdx-0x20]+        movdqu  xmm11, xmmword ptr [r11+rdx-0x20]+        movdqa  xmm12, xmm8+        punpckldq xmm8, xmm9+        punpckhdq xmm12, xmm9+        movdqa  xmm14, xmm10+        punpckldq xmm10, xmm11+        punpckhdq xmm14, xmm11+        movdqa  xmm9, xmm8+        punpcklqdq xmm8, xmm10+        punpckhqdq xmm9, xmm10+        movdqa  xmm13, xmm12+        punpcklqdq xmm12, xmm14+        punpckhqdq xmm13, xmm14+        movdqa  xmmword ptr [rsp+0x80], xmm8+        movdqa  xmmword ptr [rsp+0x90], xmm9+        movdqa  xmmword ptr [rsp+0xA0], xmm12+        movdqa  xmmword ptr [rsp+0xB0], xmm13+        movdqu  xmm8, xmmword ptr [r8+rdx-0x10]+        movdqu  xmm9, xmmword ptr [r9+rdx-0x10]+        movdqu  xmm10, xmmword ptr [r10+rdx-0x10]+        movdqu  xmm11, xmmword ptr [r11+rdx-0x10]+        movdqa  xmm12, xmm8+        punpckldq xmm8, xmm9+        punpckhdq xmm12, xmm9+        movdqa  xmm14, xmm10+        punpckldq xmm10, xmm11+        punpckhdq xmm14, xmm11+        movdqa  xmm9, xmm8+        punpcklqdq xmm8, xmm10+        punpckhqdq xmm9, xmm10+        movdqa  xmm13, xmm12+        punpcklqdq xmm12, xmm14+        punpckhqdq xmm13, xmm14+        movdqa  xmmword ptr [rsp+0xC0], xmm8+        movdqa  xmmword ptr [rsp+0xD0], xmm9+        movdqa  xmmword ptr [rsp+0xE0], xmm12+        movdqa  xmmword ptr [rsp+0xF0], xmm13+        movdqa  xmm9, xmmword ptr [BLAKE3_IV_1+rip]+        movdqa  xmm10, xmmword ptr [BLAKE3_IV_2+rip]+        movdqa  xmm11, xmmword ptr [BLAKE3_IV_3+rip]+        movdqa  xmm12, xmmword ptr [rsp+0x110]+        movdqa  xmm13, xmmword ptr [rsp+0x120]+        movdqa  xmm14, xmmword ptr [BLAKE3_BLOCK_LEN+rip]+        movd    xmm15, eax+        pshufd  xmm15, xmm15, 0x00+        prefetcht0 [r8+rdx+0x80]+        prefetcht0 [r9+rdx+0x80]+        prefetcht0 [r10+rdx+0x80]+        prefetcht0 [r11+rdx+0x80]+        paddd   xmm0, xmmword ptr [rsp]+        paddd   xmm1, xmmword ptr [rsp+0x20]+        paddd   xmm2, xmmword ptr [rsp+0x40]+        paddd   xmm3, xmmword ptr [rsp+0x60]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        movdqa  xmm8, xmmword ptr [BLAKE3_IV_0+rip]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x10]+        paddd   xmm1, xmmword ptr [rsp+0x30]+        paddd   xmm2, xmmword ptr [rsp+0x50]+        paddd   xmm3, xmmword ptr [rsp+0x70]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x80]+        paddd   xmm1, xmmword ptr [rsp+0xA0]+        paddd   xmm2, xmmword ptr [rsp+0xC0]+        paddd   xmm3, xmmword ptr [rsp+0xE0]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x90]+        paddd   xmm1, xmmword ptr [rsp+0xB0]+        paddd   xmm2, xmmword ptr [rsp+0xD0]+        paddd   xmm3, xmmword ptr [rsp+0xF0]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x20]+        paddd   xmm1, xmmword ptr [rsp+0x30]+        paddd   xmm2, xmmword ptr [rsp+0x70]+        paddd   xmm3, xmmword ptr [rsp+0x40]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x60]+        paddd   xmm1, xmmword ptr [rsp+0xA0]+        paddd   xmm2, xmmword ptr [rsp]+        paddd   xmm3, xmmword ptr [rsp+0xD0]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x10]+        paddd   xmm1, xmmword ptr [rsp+0xC0]+        paddd   xmm2, xmmword ptr [rsp+0x90]+        paddd   xmm3, xmmword ptr [rsp+0xF0]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xB0]+        paddd   xmm1, xmmword ptr [rsp+0x50]+        paddd   xmm2, xmmword ptr [rsp+0xE0]+        paddd   xmm3, xmmword ptr [rsp+0x80]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x30]+        paddd   xmm1, xmmword ptr [rsp+0xA0]+        paddd   xmm2, xmmword ptr [rsp+0xD0]+        paddd   xmm3, xmmword ptr [rsp+0x70]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x40]+        paddd   xmm1, xmmword ptr [rsp+0xC0]+        paddd   xmm2, xmmword ptr [rsp+0x20]+        paddd   xmm3, xmmword ptr [rsp+0xE0]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x60]+        paddd   xmm1, xmmword ptr [rsp+0x90]+        paddd   xmm2, xmmword ptr [rsp+0xB0]+        paddd   xmm3, xmmword ptr [rsp+0x80]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x50]+        paddd   xmm1, xmmword ptr [rsp]+        paddd   xmm2, xmmword ptr [rsp+0xF0]+        paddd   xmm3, xmmword ptr [rsp+0x10]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xA0]+        paddd   xmm1, xmmword ptr [rsp+0xC0]+        paddd   xmm2, xmmword ptr [rsp+0xE0]+        paddd   xmm3, xmmword ptr [rsp+0xD0]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x70]+        paddd   xmm1, xmmword ptr [rsp+0x90]+        paddd   xmm2, xmmword ptr [rsp+0x30]+        paddd   xmm3, xmmword ptr [rsp+0xF0]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x40]+        paddd   xmm1, xmmword ptr [rsp+0xB0]+        paddd   xmm2, xmmword ptr [rsp+0x50]+        paddd   xmm3, xmmword ptr [rsp+0x10]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp]+        paddd   xmm1, xmmword ptr [rsp+0x20]+        paddd   xmm2, xmmword ptr [rsp+0x80]+        paddd   xmm3, xmmword ptr [rsp+0x60]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xC0]+        paddd   xmm1, xmmword ptr [rsp+0x90]+        paddd   xmm2, xmmword ptr [rsp+0xF0]+        paddd   xmm3, xmmword ptr [rsp+0xE0]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xD0]+        paddd   xmm1, xmmword ptr [rsp+0xB0]+        paddd   xmm2, xmmword ptr [rsp+0xA0]+        paddd   xmm3, xmmword ptr [rsp+0x80]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x70]+        paddd   xmm1, xmmword ptr [rsp+0x50]+        paddd   xmm2, xmmword ptr [rsp]+        paddd   xmm3, xmmword ptr [rsp+0x60]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x20]+        paddd   xmm1, xmmword ptr [rsp+0x30]+        paddd   xmm2, xmmword ptr [rsp+0x10]+        paddd   xmm3, xmmword ptr [rsp+0x40]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x90]+        paddd   xmm1, xmmword ptr [rsp+0xB0]+        paddd   xmm2, xmmword ptr [rsp+0x80]+        paddd   xmm3, xmmword ptr [rsp+0xF0]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xE0]+        paddd   xmm1, xmmword ptr [rsp+0x50]+        paddd   xmm2, xmmword ptr [rsp+0xC0]+        paddd   xmm3, xmmword ptr [rsp+0x10]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xD0]+        paddd   xmm1, xmmword ptr [rsp]+        paddd   xmm2, xmmword ptr [rsp+0x20]+        paddd   xmm3, xmmword ptr [rsp+0x40]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0x30]+        paddd   xmm1, xmmword ptr [rsp+0xA0]+        paddd   xmm2, xmmword ptr [rsp+0x60]+        paddd   xmm3, xmmword ptr [rsp+0x70]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xB0]+        paddd   xmm1, xmmword ptr [rsp+0x50]+        paddd   xmm2, xmmword ptr [rsp+0x10]+        paddd   xmm3, xmmword ptr [rsp+0x80]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xF0]+        paddd   xmm1, xmmword ptr [rsp]+        paddd   xmm2, xmmword ptr [rsp+0x90]+        paddd   xmm3, xmmword ptr [rsp+0x60]+        paddd   xmm0, xmm4+        paddd   xmm1, xmm5+        paddd   xmm2, xmm6+        paddd   xmm3, xmm7+        pxor    xmm12, xmm0+        pxor    xmm13, xmm1+        pxor    xmm14, xmm2+        pxor    xmm15, xmm3+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm12+        paddd   xmm9, xmm13+        paddd   xmm10, xmm14+        paddd   xmm11, xmm15+        pxor    xmm4, xmm8+        pxor    xmm5, xmm9+        pxor    xmm6, xmm10+        pxor    xmm7, xmm11+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xE0]+        paddd   xmm1, xmmword ptr [rsp+0x20]+        paddd   xmm2, xmmword ptr [rsp+0x30]+        paddd   xmm3, xmmword ptr [rsp+0x70]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        pshuflw xmm15, xmm15, 0xB1+        pshufhw xmm15, xmm15, 0xB1+        pshuflw xmm12, xmm12, 0xB1+        pshufhw xmm12, xmm12, 0xB1+        pshuflw xmm13, xmm13, 0xB1+        pshufhw xmm13, xmm13, 0xB1+        pshuflw xmm14, xmm14, 0xB1+        pshufhw xmm14, xmm14, 0xB1+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        movdqa  xmmword ptr [rsp+0x100], xmm8+        movdqa  xmm8, xmm5+        psrld   xmm8, 12+        pslld   xmm5, 20+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 12+        pslld   xmm6, 20+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 12+        pslld   xmm7, 20+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 12+        pslld   xmm4, 20+        por     xmm4, xmm8+        paddd   xmm0, xmmword ptr [rsp+0xA0]+        paddd   xmm1, xmmword ptr [rsp+0xC0]+        paddd   xmm2, xmmword ptr [rsp+0x40]+        paddd   xmm3, xmmword ptr [rsp+0xD0]+        paddd   xmm0, xmm5+        paddd   xmm1, xmm6+        paddd   xmm2, xmm7+        paddd   xmm3, xmm4+        pxor    xmm15, xmm0+        pxor    xmm12, xmm1+        pxor    xmm13, xmm2+        pxor    xmm14, xmm3+        movdqa  xmm8, xmm15+        psrld   xmm15, 8+        pslld   xmm8, 24+        pxor    xmm15, xmm8+        movdqa  xmm8, xmm12+        psrld   xmm12, 8+        pslld   xmm8, 24+        pxor    xmm12, xmm8+        movdqa  xmm8, xmm13+        psrld   xmm13, 8+        pslld   xmm8, 24+        pxor    xmm13, xmm8+        movdqa  xmm8, xmm14+        psrld   xmm14, 8+        pslld   xmm8, 24+        pxor    xmm14, xmm8+        paddd   xmm10, xmm15+        paddd   xmm11, xmm12+        movdqa  xmm8, xmmword ptr [rsp+0x100]+        paddd   xmm8, xmm13+        paddd   xmm9, xmm14+        pxor    xmm5, xmm10+        pxor    xmm6, xmm11+        pxor    xmm7, xmm8+        pxor    xmm4, xmm9+        pxor    xmm0, xmm8+        pxor    xmm1, xmm9+        pxor    xmm2, xmm10+        pxor    xmm3, xmm11+        movdqa  xmm8, xmm5+        psrld   xmm8, 7+        pslld   xmm5, 25+        por     xmm5, xmm8+        movdqa  xmm8, xmm6+        psrld   xmm8, 7+        pslld   xmm6, 25+        por     xmm6, xmm8+        movdqa  xmm8, xmm7+        psrld   xmm8, 7+        pslld   xmm7, 25+        por     xmm7, xmm8+        movdqa  xmm8, xmm4+        psrld   xmm8, 7+        pslld   xmm4, 25+        por     xmm4, xmm8+        pxor    xmm4, xmm12+        pxor    xmm5, xmm13+        pxor    xmm6, xmm14+        pxor    xmm7, xmm15+        mov     eax, r13d+        jne     9b+        movdqa  xmm9, xmm0+        punpckldq xmm0, xmm1+        punpckhdq xmm9, xmm1+        movdqa  xmm11, xmm2+        punpckldq xmm2, xmm3+        punpckhdq xmm11, xmm3+        movdqa  xmm1, xmm0+        punpcklqdq xmm0, xmm2+        punpckhqdq xmm1, xmm2+        movdqa  xmm3, xmm9+        punpcklqdq xmm9, xmm11+        punpckhqdq xmm3, xmm11+        movdqu  xmmword ptr [rbx], xmm0+        movdqu  xmmword ptr [rbx+0x20], xmm1+        movdqu  xmmword ptr [rbx+0x40], xmm9+        movdqu  xmmword ptr [rbx+0x60], xmm3+        movdqa  xmm9, xmm4+        punpckldq xmm4, xmm5+        punpckhdq xmm9, xmm5+        movdqa  xmm11, xmm6+        punpckldq xmm6, xmm7+        punpckhdq xmm11, xmm7+        movdqa  xmm5, xmm4+        punpcklqdq xmm4, xmm6+        punpckhqdq xmm5, xmm6+        movdqa  xmm7, xmm9+        punpcklqdq xmm9, xmm11+        punpckhqdq xmm7, xmm11+        movdqu  xmmword ptr [rbx+0x10], xmm4+        movdqu  xmmword ptr [rbx+0x30], xmm5+        movdqu  xmmword ptr [rbx+0x50], xmm9+        movdqu  xmmword ptr [rbx+0x70], xmm7+        movdqa  xmm1, xmmword ptr [rsp+0x110]+        movdqa  xmm0, xmm1+        paddd   xmm1, xmmword ptr [rsp+0x150]+        movdqa  xmmword ptr [rsp+0x110], xmm1+        pxor    xmm0, xmmword ptr [CMP_MSB_MASK+rip]+        pxor    xmm1, xmmword ptr [CMP_MSB_MASK+rip]+        pcmpgtd xmm0, xmm1+        movdqa  xmm1, xmmword ptr [rsp+0x120]+        psubd   xmm1, xmm0+        movdqa  xmmword ptr [rsp+0x120], xmm1+        add     rbx, 128+        add     rdi, 32+        sub     rsi, 4+        cmp     rsi, 4+        jnc     2b+        test    rsi, rsi+        jne     3f+4:+        movdqa  xmm6, xmmword ptr [rsp+0x170]+        movdqa  xmm7, xmmword ptr [rsp+0x180]+        movdqa  xmm8, xmmword ptr [rsp+0x190]+        movdqa  xmm9, xmmword ptr [rsp+0x1A0]+        movdqa  xmm10, xmmword ptr [rsp+0x1B0]+        movdqa  xmm11, xmmword ptr [rsp+0x1C0]+        movdqa  xmm12, xmmword ptr [rsp+0x1D0]+        movdqa  xmm13, xmmword ptr [rsp+0x1E0]+        movdqa  xmm14, xmmword ptr [rsp+0x1F0]+        movdqa  xmm15, xmmword ptr [rsp+0x200]+        mov     rsp, rbp+        pop     rbp+        pop     rbx+        pop     rdi+        pop     rsi+        pop     r12+        pop     r13+        pop     r14+        pop     r15+        ret+.p2align 5+3:+        test    esi, 0x2+        je      3f+        movups  xmm0, xmmword ptr [rcx]+        movups  xmm1, xmmword ptr [rcx+0x10]+        movaps  xmm8, xmm0+        movaps  xmm9, xmm1+        movd    xmm13, dword ptr [rsp+0x110]+        movd    xmm14, dword ptr [rsp+0x120]+        punpckldq xmm13, xmm14+        movaps  xmmword ptr [rsp], xmm13+        movd    xmm14, dword ptr [rsp+0x114]+        movd    xmm13, dword ptr [rsp+0x124]+        punpckldq xmm14, xmm13+        movaps  xmmword ptr [rsp+0x10], xmm14+        mov     r8, qword ptr [rdi]+        mov     r9, qword ptr [rdi+0x8]+        movzx   eax, byte ptr [rbp+0x80]+        or      eax, r13d+        xor     edx, edx+2:+        mov     r14d, eax+        or      eax, r12d+        add     rdx, 64+        cmp     rdx, r15+        cmovne  eax, r14d+        movaps  xmm2, xmmword ptr [BLAKE3_IV+rip]+        movaps  xmm10, xmm2+        movups  xmm4, xmmword ptr [r8+rdx-0x40]+        movups  xmm5, xmmword ptr [r8+rdx-0x30]+        movaps  xmm3, xmm4+        shufps  xmm4, xmm5, 136+        shufps  xmm3, xmm5, 221+        movaps  xmm5, xmm3+        movups  xmm6, xmmword ptr [r8+rdx-0x20]+        movups  xmm7, xmmword ptr [r8+rdx-0x10]+        movaps  xmm3, xmm6+        shufps  xmm6, xmm7, 136+        pshufd  xmm6, xmm6, 0x93+        shufps  xmm3, xmm7, 221+        pshufd  xmm7, xmm3, 0x93+        movups  xmm12, xmmword ptr [r9+rdx-0x40]+        movups  xmm13, xmmword ptr [r9+rdx-0x30]+        movaps  xmm11, xmm12+        shufps  xmm12, xmm13, 136+        shufps  xmm11, xmm13, 221+        movaps  xmm13, xmm11+        movups  xmm14, xmmword ptr [r9+rdx-0x20]+        movups  xmm15, xmmword ptr [r9+rdx-0x10]+        movaps  xmm11, xmm14+        shufps  xmm14, xmm15, 136+        pshufd  xmm14, xmm14, 0x93+        shufps  xmm11, xmm15, 221+        pshufd  xmm15, xmm11, 0x93+        shl     rax, 0x20+        or      rax, 0x40+        movq    xmm3, rax+        movdqa  xmmword ptr [rsp+0x20], xmm3+        movaps  xmm3, xmmword ptr [rsp]+        movaps  xmm11, xmmword ptr [rsp+0x10]+        punpcklqdq xmm3, xmmword ptr [rsp+0x20]+        punpcklqdq xmm11, xmmword ptr [rsp+0x20]+        mov     al, 7+9:+        paddd   xmm0, xmm4+        paddd   xmm8, xmm12+        movaps  xmmword ptr [rsp+0x20], xmm4+        movaps  xmmword ptr [rsp+0x30], xmm12+        paddd   xmm0, xmm1+        paddd   xmm8, xmm9+        pxor    xmm3, xmm0+        pxor    xmm11, xmm8+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        pshuflw xmm11, xmm11, 0xB1+        pshufhw xmm11, xmm11, 0xB1+        paddd   xmm2, xmm3+        paddd   xmm10, xmm11+        pxor    xmm1, xmm2+        pxor    xmm9, xmm10+        movdqa  xmm4, xmm1+        pslld   xmm1, 20+        psrld   xmm4, 12+        por     xmm1, xmm4+        movdqa  xmm4, xmm9+        pslld   xmm9, 20+        psrld   xmm4, 12+        por     xmm9, xmm4+        paddd   xmm0, xmm5+        paddd   xmm8, xmm13+        movaps  xmmword ptr [rsp+0x40], xmm5+        movaps  xmmword ptr [rsp+0x50], xmm13+        paddd   xmm0, xmm1+        paddd   xmm8, xmm9+        pxor    xmm3, xmm0+        pxor    xmm11, xmm8+        movdqa  xmm13, xmm3+        psrld   xmm3, 8+        pslld   xmm13, 24+        pxor    xmm3, xmm13+        movdqa  xmm13, xmm11+        psrld   xmm11, 8+        pslld   xmm13, 24+        pxor    xmm11, xmm13+        paddd   xmm2, xmm3+        paddd   xmm10, xmm11+        pxor    xmm1, xmm2+        pxor    xmm9, xmm10+        movdqa  xmm4, xmm1+        pslld   xmm1, 25+        psrld   xmm4, 7+        por     xmm1, xmm4+        movdqa  xmm4, xmm9+        pslld   xmm9, 25+        psrld   xmm4, 7+        por     xmm9, xmm4+        pshufd  xmm0, xmm0, 0x93+        pshufd  xmm8, xmm8, 0x93+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm11, xmm11, 0x4E+        pshufd  xmm2, xmm2, 0x39+        pshufd  xmm10, xmm10, 0x39+        paddd   xmm0, xmm6+        paddd   xmm8, xmm14+        paddd   xmm0, xmm1+        paddd   xmm8, xmm9+        pxor    xmm3, xmm0+        pxor    xmm11, xmm8+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        pshuflw xmm11, xmm11, 0xB1+        pshufhw xmm11, xmm11, 0xB1+        paddd   xmm2, xmm3+        paddd   xmm10, xmm11+        pxor    xmm1, xmm2+        pxor    xmm9, xmm10+        movdqa  xmm4, xmm1+        pslld   xmm1, 20+        psrld   xmm4, 12+        por     xmm1, xmm4+        movdqa  xmm4, xmm9+        pslld   xmm9, 20+        psrld   xmm4, 12+        por     xmm9, xmm4+        paddd   xmm0, xmm7+        paddd   xmm8, xmm15+        paddd   xmm0, xmm1+        paddd   xmm8, xmm9+        pxor    xmm3, xmm0+        pxor    xmm11, xmm8+        movdqa  xmm13, xmm3+        psrld   xmm3, 8+        pslld   xmm13, 24+        pxor    xmm3, xmm13+        movdqa  xmm13, xmm11+        psrld   xmm11, 8+        pslld   xmm13, 24+        pxor    xmm11, xmm13+        paddd   xmm2, xmm3+        paddd   xmm10, xmm11+        pxor    xmm1, xmm2+        pxor    xmm9, xmm10+        movdqa  xmm4, xmm1+        pslld   xmm1, 25+        psrld   xmm4, 7+        por     xmm1, xmm4+        movdqa  xmm4, xmm9+        pslld   xmm9, 25+        psrld   xmm4, 7+        por     xmm9, xmm4+        pshufd  xmm0, xmm0, 0x39+        pshufd  xmm8, xmm8, 0x39+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm11, xmm11, 0x4E+        pshufd  xmm2, xmm2, 0x93+        pshufd  xmm10, xmm10, 0x93+        dec     al+        je      9f+        movdqa  xmm12, xmmword ptr [rsp+0x20]+        movdqa  xmm5, xmmword ptr [rsp+0x40]+        pshufd  xmm13, xmm12, 0x0F+        shufps  xmm12, xmm5, 214+        pshufd  xmm4, xmm12, 0x39+        movdqa  xmm12, xmm6+        shufps  xmm12, xmm7, 250+        pand    xmm13, xmmword ptr [PBLENDW_0x33_MASK+rip]+        pand    xmm12, xmmword ptr [PBLENDW_0xCC_MASK+rip]+        por     xmm13, xmm12+        movdqa  xmmword ptr [rsp+0x20], xmm13+        movdqa  xmm12, xmm7+        punpcklqdq xmm12, xmm5+        movdqa  xmm13, xmm6+        pand    xmm12, xmmword ptr [PBLENDW_0x3F_MASK+rip]+        pand    xmm13, xmmword ptr [PBLENDW_0xC0_MASK+rip]+        por     xmm12, xmm13+        pshufd  xmm12, xmm12, 0x78+        punpckhdq xmm5, xmm7+        punpckldq xmm6, xmm5+        pshufd  xmm7, xmm6, 0x1E+        movdqa  xmmword ptr [rsp+0x40], xmm12+        movdqa  xmm5, xmmword ptr [rsp+0x30]+        movdqa  xmm13, xmmword ptr [rsp+0x50]+        pshufd  xmm6, xmm5, 0x0F+        shufps  xmm5, xmm13, 214+        pshufd  xmm12, xmm5, 0x39+        movdqa  xmm5, xmm14+        shufps  xmm5, xmm15, 250+        pand    xmm6, xmmword ptr [PBLENDW_0x33_MASK+rip]+        pand    xmm5, xmmword ptr [PBLENDW_0xCC_MASK+rip]+        por     xmm6, xmm5+        movdqa  xmm5, xmm15+        punpcklqdq xmm5, xmm13+        movdqa  xmmword ptr [rsp+0x30], xmm2+        movdqa  xmm2, xmm14+        pand    xmm5, xmmword ptr [PBLENDW_0x3F_MASK+rip]+        pand    xmm2, xmmword ptr [PBLENDW_0xC0_MASK+rip]+        por     xmm5, xmm2+        movdqa  xmm2, xmmword ptr [rsp+0x30]+        pshufd  xmm5, xmm5, 0x78+        punpckhdq xmm13, xmm15+        punpckldq xmm14, xmm13+        pshufd  xmm15, xmm14, 0x1E+        movdqa  xmm13, xmm6+        movdqa  xmm14, xmm5+        movdqa  xmm5, xmmword ptr [rsp+0x20]+        movdqa  xmm6, xmmword ptr [rsp+0x40]+        jmp     9b+9:+        pxor    xmm0, xmm2+        pxor    xmm1, xmm3+        pxor    xmm8, xmm10+        pxor    xmm9, xmm11+        mov     eax, r13d+        cmp     rdx, r15+        jne     2b+        movups  xmmword ptr [rbx], xmm0+        movups  xmmword ptr [rbx+0x10], xmm1+        movups  xmmword ptr [rbx+0x20], xmm8+        movups  xmmword ptr [rbx+0x30], xmm9+        mov     eax, dword ptr [rsp+0x130]+        neg     eax+        mov    r10d, dword ptr [rsp+0x110+8*rax]+        mov    r11d, dword ptr [rsp+0x120+8*rax]+        mov dword ptr [rsp+0x110], r10d+        mov dword ptr [rsp+0x120], r11d+        add     rdi, 16+        add     rbx, 64+        sub     rsi, 2+3:+        test    esi, 0x1+        je      4b+        movups  xmm0, xmmword ptr [rcx]+        movups  xmm1, xmmword ptr [rcx+0x10]+        movd    xmm13, dword ptr [rsp+0x110]+        movd    xmm14, dword ptr [rsp+0x120]+        punpckldq xmm13, xmm14+        mov     r8, qword ptr [rdi]+        movzx   eax, byte ptr [rbp+0x80]+        or      eax, r13d+        xor     edx, edx+2:+        mov     r14d, eax+        or      eax, r12d+        add     rdx, 64+        cmp     rdx, r15+        cmovne  eax, r14d+        movaps  xmm2, xmmword ptr [BLAKE3_IV+rip]+        shl     rax, 32+        or      rax, 64+        movq    xmm12, rax+        movdqa  xmm3, xmm13+        punpcklqdq xmm3, xmm12+        movups  xmm4, xmmword ptr [r8+rdx-0x40]+        movups  xmm5, xmmword ptr [r8+rdx-0x30]+        movaps  xmm8, xmm4+        shufps  xmm4, xmm5, 136+        shufps  xmm8, xmm5, 221+        movaps  xmm5, xmm8+        movups  xmm6, xmmword ptr [r8+rdx-0x20]+        movups  xmm7, xmmword ptr [r8+rdx-0x10]+        movaps  xmm8, xmm6+        shufps  xmm6, xmm7, 136+        pshufd  xmm6, xmm6, 0x93+        shufps  xmm8, xmm7, 221+        pshufd  xmm7, xmm8, 0x93+        mov     al, 7+9:+        paddd   xmm0, xmm4+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 20+        psrld   xmm11, 12+        por     xmm1, xmm11+        paddd   xmm0, xmm5+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        movdqa  xmm14, xmm3+        psrld   xmm3, 8+        pslld   xmm14, 24+        pxor    xmm3, xmm14+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 25+        psrld   xmm11, 7+        por     xmm1, xmm11+        pshufd  xmm0, xmm0, 0x93+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm2, xmm2, 0x39+        paddd   xmm0, xmm6+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 20+        psrld   xmm11, 12+        por     xmm1, xmm11+        paddd   xmm0, xmm7+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        movdqa  xmm14, xmm3+        psrld   xmm3, 8+        pslld   xmm14, 24+        pxor    xmm3, xmm14+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 25+        psrld   xmm11, 7+        por     xmm1, xmm11+        pshufd  xmm0, xmm0, 0x39+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm2, xmm2, 0x93+        dec     al+        jz      9f+        movdqa  xmm8, xmm4+        shufps  xmm8, xmm5, 214+        pshufd  xmm9, xmm4, 0x0F+        pshufd  xmm4, xmm8, 0x39+        movdqa  xmm8, xmm6+        shufps  xmm8, xmm7, 250+        pand    xmm9, xmmword ptr [PBLENDW_0x33_MASK+rip]+        pand    xmm8, xmmword ptr [PBLENDW_0xCC_MASK+rip]+        por     xmm9, xmm8+        movdqa  xmm8, xmm7+        punpcklqdq xmm8, xmm5+        movdqa  xmm10, xmm6+        pand    xmm8, xmmword ptr [PBLENDW_0x3F_MASK+rip]+        pand    xmm10, xmmword ptr [PBLENDW_0xC0_MASK+rip]+        por     xmm8, xmm10+        pshufd  xmm8, xmm8, 0x78+        punpckhdq xmm5, xmm7+        punpckldq xmm6, xmm5+        pshufd  xmm7, xmm6, 0x1E+        movdqa  xmm5, xmm9+        movdqa  xmm6, xmm8+        jmp     9b+9:+        pxor    xmm0, xmm2+        pxor    xmm1, xmm3+        mov     eax, r13d+        cmp     rdx, r15+        jne     2b+        movups  xmmword ptr [rbx], xmm0+        movups  xmmword ptr [rbx+0x10], xmm1+        jmp     4b++.p2align 6+blake3_compress_in_place_sse2:+_blake3_compress_in_place_sse2:+        sub     rsp, 120+        movdqa  xmmword ptr [rsp], xmm6+        movdqa  xmmword ptr [rsp+0x10], xmm7+        movdqa  xmmword ptr [rsp+0x20], xmm8+        movdqa  xmmword ptr [rsp+0x30], xmm9+        movdqa  xmmword ptr [rsp+0x40], xmm11+        movdqa  xmmword ptr [rsp+0x50], xmm14+        movdqa  xmmword ptr [rsp+0x60], xmm15+        movups  xmm0, xmmword ptr [rcx]+        movups  xmm1, xmmword ptr [rcx+0x10]+        movaps  xmm2, xmmword ptr [BLAKE3_IV+rip]+        movzx   eax, byte ptr [rsp+0xA0]+        movzx   r8d, r8b+        shl     rax, 32+        add     r8, rax+        movq    xmm3, r9+        movq    xmm4, r8+        punpcklqdq xmm3, xmm4+        movups  xmm4, xmmword ptr [rdx]+        movups  xmm5, xmmword ptr [rdx+0x10]+        movaps  xmm8, xmm4+        shufps  xmm4, xmm5, 136+        shufps  xmm8, xmm5, 221+        movaps  xmm5, xmm8+        movups  xmm6, xmmword ptr [rdx+0x20]+        movups  xmm7, xmmword ptr [rdx+0x30]+        movaps  xmm8, xmm6+        shufps  xmm6, xmm7, 136+        pshufd  xmm6, xmm6, 0x93+        shufps  xmm8, xmm7, 221+        pshufd  xmm7, xmm8, 0x93+        mov     al, 7+9:+        paddd   xmm0, xmm4+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 20+        psrld   xmm11, 12+        por     xmm1, xmm11+        paddd   xmm0, xmm5+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        movdqa  xmm14, xmm3+        psrld   xmm3, 8+        pslld   xmm14, 24+        pxor    xmm3, xmm14+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 25+        psrld   xmm11, 7+        por     xmm1, xmm11+        pshufd  xmm0, xmm0, 0x93+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm2, xmm2, 0x39+        paddd   xmm0, xmm6+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 20+        psrld   xmm11, 12+        por     xmm1, xmm11+        paddd   xmm0, xmm7+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        movdqa  xmm14, xmm3+        psrld   xmm3, 8+        pslld   xmm14, 24+        pxor    xmm3, xmm14+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 25+        psrld   xmm11, 7+        por     xmm1, xmm11+        pshufd  xmm0, xmm0, 0x39+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm2, xmm2, 0x93+        dec     al+        jz      9f+        movdqa  xmm8, xmm4+        shufps  xmm8, xmm5, 214+        pshufd  xmm9, xmm4, 0x0F+        pshufd  xmm4, xmm8, 0x39+        movdqa  xmm8, xmm6+        shufps  xmm8, xmm7, 250+        pand    xmm9, xmmword ptr [PBLENDW_0x33_MASK+rip]+        pand    xmm8, xmmword ptr [PBLENDW_0xCC_MASK+rip]+        por     xmm9, xmm8+        movdqa  xmm8, xmm7+        punpcklqdq xmm8, xmm5+        movdqa  xmm14, xmm6+        pand    xmm8, xmmword ptr [PBLENDW_0x3F_MASK+rip]+        pand    xmm14, xmmword ptr [PBLENDW_0xC0_MASK+rip]+        por     xmm8, xmm14+        pshufd  xmm8, xmm8, 0x78+        punpckhdq xmm5, xmm7+        punpckldq xmm6, xmm5+        pshufd  xmm7, xmm6, 0x1E+        movdqa  xmm5, xmm9+        movdqa  xmm6, xmm8+        jmp     9b+9:+        pxor    xmm0, xmm2+        pxor    xmm1, xmm3+        movups  xmmword ptr [rcx], xmm0+        movups  xmmword ptr [rcx+0x10], xmm1+        movdqa  xmm6, xmmword ptr [rsp]+        movdqa  xmm7, xmmword ptr [rsp+0x10]+        movdqa  xmm8, xmmword ptr [rsp+0x20]+        movdqa  xmm9, xmmword ptr [rsp+0x30]+        movdqa  xmm11, xmmword ptr [rsp+0x40]+        movdqa  xmm14, xmmword ptr [rsp+0x50]+        movdqa  xmm15, xmmword ptr [rsp+0x60]+        add     rsp, 120+        ret+++.p2align 6+_blake3_compress_xof_sse2:+blake3_compress_xof_sse2:+        sub     rsp, 120+        movdqa  xmmword ptr [rsp], xmm6+        movdqa  xmmword ptr [rsp+0x10], xmm7+        movdqa  xmmword ptr [rsp+0x20], xmm8+        movdqa  xmmword ptr [rsp+0x30], xmm9+        movdqa  xmmword ptr [rsp+0x40], xmm11+        movdqa  xmmword ptr [rsp+0x50], xmm14+        movdqa  xmmword ptr [rsp+0x60], xmm15+        movups  xmm0, xmmword ptr [rcx]+        movups  xmm1, xmmword ptr [rcx+0x10]+        movaps  xmm2, xmmword ptr [BLAKE3_IV+rip]+        movzx   eax, byte ptr [rsp+0xA0]+        movzx   r8d, r8b+        mov     r10, qword ptr [rsp+0xA8]+        shl     rax, 32+        add     r8, rax+        movq    xmm3, r9+        movq    xmm4, r8+        punpcklqdq xmm3, xmm4+        movups  xmm4, xmmword ptr [rdx]+        movups  xmm5, xmmword ptr [rdx+0x10]+        movaps  xmm8, xmm4+        shufps  xmm4, xmm5, 136+        shufps  xmm8, xmm5, 221+        movaps  xmm5, xmm8+        movups  xmm6, xmmword ptr [rdx+0x20]+        movups  xmm7, xmmword ptr [rdx+0x30]+        movaps  xmm8, xmm6+        shufps  xmm6, xmm7, 136+        pshufd  xmm6, xmm6, 0x93+        shufps  xmm8, xmm7, 221+        pshufd  xmm7, xmm8, 0x93+        mov     al, 7+9:+        paddd   xmm0, xmm4+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 20+        psrld   xmm11, 12+        por     xmm1, xmm11+        paddd   xmm0, xmm5+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        movdqa  xmm14, xmm3+        psrld   xmm3, 8+        pslld   xmm14, 24+        pxor    xmm3, xmm14+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 25+        psrld   xmm11, 7+        por     xmm1, xmm11+        pshufd  xmm0, xmm0, 0x93+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm2, xmm2, 0x39+        paddd   xmm0, xmm6+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        pshuflw xmm3, xmm3, 0xB1+        pshufhw xmm3, xmm3, 0xB1+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 20+        psrld   xmm11, 12+        por     xmm1, xmm11+        paddd   xmm0, xmm7+        paddd   xmm0, xmm1+        pxor    xmm3, xmm0+        movdqa  xmm14, xmm3+        psrld   xmm3, 8+        pslld   xmm14, 24+        pxor    xmm3, xmm14+        paddd   xmm2, xmm3+        pxor    xmm1, xmm2+        movdqa  xmm11, xmm1+        pslld   xmm1, 25+        psrld   xmm11, 7+        por     xmm1, xmm11+        pshufd  xmm0, xmm0, 0x39+        pshufd  xmm3, xmm3, 0x4E+        pshufd  xmm2, xmm2, 0x93+        dec     al+        jz      9f+        movdqa  xmm8, xmm4+        shufps  xmm8, xmm5, 214+        pshufd  xmm9, xmm4, 0x0F+        pshufd  xmm4, xmm8, 0x39+        movdqa  xmm8, xmm6+        shufps  xmm8, xmm7, 250+        pand    xmm9, xmmword ptr [PBLENDW_0x33_MASK+rip]+        pand    xmm8, xmmword ptr [PBLENDW_0xCC_MASK+rip]+        por     xmm9, xmm8+        movdqa  xmm8, xmm7+        punpcklqdq xmm8, xmm5+        movdqa  xmm14, xmm6+        pand    xmm8, xmmword ptr [PBLENDW_0x3F_MASK+rip]+        pand    xmm14, xmmword ptr [PBLENDW_0xC0_MASK+rip]+        por     xmm8, xmm14+        pshufd  xmm8, xmm8, 0x78+        punpckhdq xmm5, xmm7+        punpckldq xmm6, xmm5+        pshufd  xmm7, xmm6, 0x1E+        movdqa  xmm5, xmm9+        movdqa  xmm6, xmm8+        jmp     9b+9:+        movdqu  xmm4, xmmword ptr [rcx]+        movdqu  xmm5, xmmword ptr [rcx+0x10]+        pxor    xmm0, xmm2+        pxor    xmm1, xmm3+        pxor    xmm2, xmm4+        pxor    xmm3, xmm5+        movups  xmmword ptr [r10], xmm0+        movups  xmmword ptr [r10+0x10], xmm1+        movups  xmmword ptr [r10+0x20], xmm2+        movups  xmmword ptr [r10+0x30], xmm3+        movdqa  xmm6, xmmword ptr [rsp]+        movdqa  xmm7, xmmword ptr [rsp+0x10]+        movdqa  xmm8, xmmword ptr [rsp+0x20]+        movdqa  xmm9, xmmword ptr [rsp+0x30]+        movdqa  xmm11, xmmword ptr [rsp+0x40]+        movdqa  xmm14, xmmword ptr [rsp+0x50]+        movdqa  xmm15, xmmword ptr [rsp+0x60]+        add     rsp, 120+        ret+++.section .rdata+.p2align  6+BLAKE3_IV:+        .long  0x6A09E667, 0xBB67AE85+        .long  0x3C6EF372, 0xA54FF53A+ADD0:   +        .long  0, 1, 2, 3+ADD1:+        .long  4, 4, 4, 4+BLAKE3_IV_0:+        .long  0x6A09E667, 0x6A09E667, 0x6A09E667, 0x6A09E667+BLAKE3_IV_1:+        .long  0xBB67AE85, 0xBB67AE85, 0xBB67AE85, 0xBB67AE85+BLAKE3_IV_2:+        .long  0x3C6EF372, 0x3C6EF372, 0x3C6EF372, 0x3C6EF372+BLAKE3_IV_3:+        .long  0xA54FF53A, 0xA54FF53A, 0xA54FF53A, 0xA54FF53A+BLAKE3_BLOCK_LEN:+        .long  64, 64, 64, 64+CMP_MSB_MASK:+        .long  0x80000000, 0x80000000, 0x80000000, 0x80000000+PBLENDW_0x33_MASK:+        .long  0xFFFFFFFF, 0x00000000, 0xFFFFFFFF, 0x00000000+PBLENDW_0xCC_MASK:+        .long  0x00000000, 0xFFFFFFFF, 0x00000000, 0xFFFFFFFF+PBLENDW_0x3F_MASK:+        .long  0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0x00000000+PBLENDW_0xC0_MASK:+        .long  0x00000000, 0x00000000, 0x00000000, 0xFFFFFFFF

@@ -2,6 +2,12 @@  #include <immintrin.h> +#if defined(__clang__)+#pragma clang attribute push (__attribute__((target("sse4.1"))), apply_to=function)+#elif defined(__GNUC__)+#pragma GCC target("sse4.1")+#endif+ #define DEGREE 4  #define _mm_shuffle_ps2(a, b, c)                                               \@@ -27,7 +33,6 @@   return _mm_setr_epi32((int32_t)a, (int32_t)b, (int32_t)c, (int32_t)d); } -TARGET_SSE41 INLINE __m128i rot16(__m128i x) {   return _mm_shuffle_epi8(       x, _mm_set_epi8(13, 12, 15, 14, 9, 8, 11, 10, 5, 4, 7, 6, 1, 0, 3, 2));@@ -37,7 +42,6 @@   return xorv(_mm_srli_epi32(x, 12), _mm_slli_epi32(x, 32 - 12)); } -TARGET_SSE41 INLINE __m128i rot8(__m128i x) {   return _mm_shuffle_epi8(       x, _mm_set_epi8(12, 15, 14, 13, 8, 11, 10, 9, 4, 7, 6, 5, 0, 3, 2, 1));@@ -47,7 +51,6 @@   return xorv(_mm_srli_epi32(x, 7), _mm_slli_epi32(x, 32 - 7)); } -TARGET_SSE41 INLINE void g1(__m128i *row0, __m128i *row1, __m128i *row2, __m128i *row3,                __m128i m) {   *row0 = addv(addv(*row0, m), *row1);@@ -58,7 +61,6 @@   *row1 = rot12(*row1); } -TARGET_SSE41 INLINE void g2(__m128i *row0, __m128i *row1, __m128i *row2, __m128i *row3,                __m128i m) {   *row0 = addv(addv(*row0, m), *row1);@@ -84,7 +86,6 @@   *row2 = _mm_shuffle_epi32(*row2, _MM_SHUFFLE(2, 1, 0, 3)); } -TARGET_SSE41 INLINE void compress_pre(__m128i rows[4], const uint32_t cv[8],                          const uint8_t block[BLAKE3_BLOCK_LEN],                          uint8_t block_len, uint64_t counter, uint8_t flags) {@@ -256,7 +257,6 @@   undiagonalize(&rows[0], &rows[2], &rows[3]); } -TARGET_SSE41 void blake3_compress_in_place_sse41(uint32_t cv[8],                                     const uint8_t block[BLAKE3_BLOCK_LEN],                                     uint8_t block_len, uint64_t counter,@@ -267,7 +267,6 @@   storeu(xorv(rows[1], rows[3]), (uint8_t *)&cv[4]); } -TARGET_SSE41 void blake3_compress_xof_sse41(const uint32_t cv[8],                                const uint8_t block[BLAKE3_BLOCK_LEN],                                uint8_t block_len, uint64_t counter,@@ -280,7 +279,6 @@   storeu(xorv(rows[3], loadu((uint8_t *)&cv[4])), &out[48]); } -TARGET_SSE41 INLINE void round_fn(__m128i v[16], __m128i m[16], size_t r) {   v[0] = addv(v[0], m[(size_t)MSG_SCHEDULE[r][0]]);   v[1] = addv(v[1], m[(size_t)MSG_SCHEDULE[r][2]]);@@ -437,7 +435,7 @@   out[14] = loadu(&inputs[2][block_offset + 3 * sizeof(__m128i)]);   out[15] = loadu(&inputs[3][block_offset + 3 * sizeof(__m128i)]);   for (size_t i = 0; i < 4; ++i) {-    _mm_prefetch(&inputs[i][block_offset + 256], _MM_HINT_T0);+    _mm_prefetch((const void *)&inputs[i][block_offset + 256], _MM_HINT_T0);   }   transpose_vecs(&out[0]);   transpose_vecs(&out[4]);@@ -450,15 +448,15 @@   const __m128i mask = _mm_set1_epi32(-(int32_t)increment_counter);   const __m128i add0 = _mm_set_epi32(3, 2, 1, 0);   const __m128i add1 = _mm_and_si128(mask, add0);-  __m128i l = _mm_add_epi32(_mm_set1_epi32(counter), add1);+  __m128i l = _mm_add_epi32(_mm_set1_epi32((int32_t)counter), add1);   __m128i carry = _mm_cmpgt_epi32(_mm_xor_si128(add1, _mm_set1_epi32(0x80000000)),                                   _mm_xor_si128(   l, _mm_set1_epi32(0x80000000)));-  __m128i h = _mm_sub_epi32(_mm_set1_epi32(counter >> 32), carry);+  __m128i h = _mm_sub_epi32(_mm_set1_epi32((int32_t)(counter >> 32)), carry);   *out_lo = l;   *out_hi = h; } -TARGET_SSE41+static void blake3_hash4_sse41(const uint8_t *const *inputs, size_t blocks,                         const uint32_t key[8], uint64_t counter,                         bool increment_counter, uint8_t flags,@@ -566,3 +564,8 @@     out = &out[BLAKE3_OUT_LEN];   } }++#if defined(__clang__)+#pragma clang attribute pop+#endif+

@@ -1,3 +1,17 @@+#if defined(__ELF__) && defined(__linux__)+.section .note.GNU-stack,"",%progbits+#endif++#if defined(__ELF__) && defined(__CET__) && defined(__has_include)+#if __has_include(<cet.h>)+#include <cet.h>+#endif+#endif++#if !defined(_CET_ENDBR)+#define _CET_ENDBR+#endif+ .intel_syntax noprefix .global blake3_hash_many_sse41 .global _blake3_hash_many_sse41@@ -13,6 +27,7 @@         .p2align  6 _blake3_hash_many_sse41: blake3_hash_many_sse41:+        _CET_ENDBR         push    r15         push    r14         push    r13@@ -1774,6 +1789,7 @@ .p2align 6 blake3_compress_in_place_sse41: _blake3_compress_in_place_sse41:+        _CET_ENDBR         movups  xmm0, xmmword ptr [rdi]         movups  xmm1, xmmword ptr [rdi+0x10]         movaps  xmm2, xmmword ptr [BLAKE3_IV+rip]@@ -1874,6 +1890,7 @@ .p2align 6 blake3_compress_xof_sse41: _blake3_compress_xof_sse41:+        _CET_ENDBR         movups  xmm0, xmmword ptr [rdi]         movups  xmm1, xmmword ptr [rdi+0x10]         movaps  xmm2, xmmword ptr [BLAKE3_IV+rip]

@@ -2042,7 +2042,7 @@         ret  -.section .rodata+.section .rdata .p2align  6 BLAKE3_IV:         .long  0x6A09E667, 0xBB67AE85

@@ -7,7 +7,7 @@  -- | Haskell bindings to the fast [official BLAKE3 hashing -- implementation in assembly and C](https://github.com/BLAKE3-team/BLAKE3).--- With support for AVX-512, AVX2 and SSE 4.1.+-- With support for AVX-512, AVX2, SSE 2, and SSE 4.1. -- -- The original assembly and C implementation is released into the public domain with CC0 1.0. -- Alternatively, it is licensed under the Apache License 2.0, copyright of Jack@@ -20,20 +20,15 @@ module BLAKE3   ( -- * Hashing     hash-  , BIO.Digest-  , BIO.digest+  , BIO.Digest(..)     -- * Keyed hashing-  , hashKeyed   , BIO.Key   , BIO.key     -- * Key derivation   , derive-  , BIO.Context-  , BIO.context     -- * Incremental hashing   , BIO.Hasher-  , hasher-  , hasherKeyed+  , init   , update   , finalize   , finalizeSeek@@ -48,7 +43,7 @@ import qualified Data.ByteArray.Sized as BAS import Data.Proxy import Data.Word-import GHC.TypeLits+import Prelude hiding (init) import System.IO.Unsafe (unsafeDupablePerformIO)  import qualified BLAKE3.IO as BIO@@ -57,58 +52,46 @@  -- | BLAKE3 hashing. ----- For incremental hashing, see 'hasher', 'update' and 'finalize':+-- For incremental hashing, see 'init', 'update' and 'finalize': -- -- @--- 'hash' = 'finalize' '.' 'update' 'hasher'+-- 'hash' yk = 'finalize' '.' 'update' ('init' yk) -- @ hash-  :: forall len bin-  .  (KnownNat len, BA.ByteArrayAccess bin)-  => [bin]           -- ^ Data to hash.-  -> BIO.Digest len-  -- ^ Default digest length is 'BIO.DEFAULT_DIGEST_LEN'.-  -- The 'Digest' is wiped from memory as soon as the 'Digest' becomes unused.-hash = unsafeDupablePerformIO . BIO.hash+  :: forall len digest bin+  .  (BAS.ByteArrayN len digest, BA.ByteArrayAccess bin)+  => Maybe BIO.Key -- ^ Whether to use keyed hashing mode (for MAC, PRF).+  -> [bin]  -- ^ Data to hash.+  -> digest -- ^ The @digest@ type could be @'BIO.Digest' len@.+hash yk = unsafeDupablePerformIO . BIO.hash yk {-# NOINLINE hash #-} --- | BLAKE3 hashing with a 'BIO.Key'.+-- | BLAKE3 key derivation. ----- This can be used for MAC (message authentication code), PRF (pseudo random--- function) and SHO (stateful hash object) purposes.+-- This can be used for KDF (key derivation function) purposes. ----- For incremental hashing, see 'hasherKeyed', 'update' and 'finalize':+-- The key derivation @context@ should be hardcoded, globally unique,+-- application-specific well-known string. --+-- A good format for the context string is:+-- -- @--- 'hashKeyed' key = 'finalize' '.' 'update' ('hasherKeyed' key)+-- [application] [commit timestamp] [purpose] -- @-hashKeyed-  :: forall len bin-  .  (KnownNat len, BA.ByteArrayAccess bin)-  => BIO.Key-  -> [bin]           -- ^ Data to hash.-  -> BIO.Digest len-  -- ^ Default digest length is 'BIO.DEFAULT_DIGEST_LEN'.-  -- The 'Digest' is wiped from memory as soon as the 'Digest' becomes unused.-hashKeyed key0 bins = unsafeDupablePerformIO $ do-  (dig, _ :: BIO.Hasher) <- BAS.allocRet Proxy $ \ph -> do-    BIO.initKeyed ph key0-    BIO.update ph bins-    BIO.finalize ph-  pure dig-{-# NOINLINE hashKeyed #-}---- | BLAKE3 key derivation. ----- This can be used for KDF (key derivation function) purposes.+-- For example:+--+-- @+-- example.com 2019-12-25 16:18:03 session tokens v1+-- @ derive-  :: forall len ikm-  .  (KnownNat len, BA.ByteArrayAccess ikm)-  => BIO.Context-  -> [ikm]  -- ^ Input key material.-  -> BIO.Digest len-  -- ^ Default digest length is 'BIO.DEFAULT_DIGEST_LEN'.-  -- The 'Digest' is wiped from memory as soon as the 'Digest' becomes unused.+  :: forall len okm ikm context+  .  (BAS.ByteArrayN len okm,+      BA.ByteArrayAccess ikm,+      BA.ByteArrayAccess context)+  => context -- ^ Key derivation context.+  -> [ikm]   -- ^ Input key material.+  -> okm     -- ^ Output key material of the specified @len@ght. derive ctx ikms = unsafeDupablePerformIO $ do   (dig, _ :: BIO.Hasher) <- BAS.allocRet Proxy $ \ph -> do     BIO.initDerive ph ctx@@ -118,14 +101,12 @@ {-# NOINLINE derive #-}  -- | Initial 'BIO.Hasher' for incremental hashing.-hasher :: BIO.Hasher -- ^-hasher = BAS.allocAndFreeze BIO.init---- | Initial 'BIO.Hasher' for incremental /keyed/ hashing.-hasherKeyed :: BIO.Key -> BIO.Hasher -- ^-hasherKeyed key0 =+init+  :: Maybe BIO.Key -- ^ Whether to use keyed hashing mode (for MAC, PRF).+  -> BIO.Hasher+init yk =   BAS.allocAndFreeze $ \ph ->-  BIO.initKeyed ph key0+  BIO.init ph yk  -- | Update 'BIO.Hasher' with new data. update@@ -138,37 +119,31 @@   BAS.copyAndFreeze h0 $ \ph1 ->   BIO.update ph1 bins --- | Finish hashing and obtain a 'BIO.Digest' of the specified @len@gth.+-- | Finalize incremental hashing and obtain a the BLAKE3 output of the+-- specified @len@gth. finalize-  :: forall len-  .  KnownNat len+  :: forall len output+  .  BAS.ByteArrayN len output   => BIO.Hasher-  -> BIO.Digest len-  -- ^ Default digest length is 'BIO.DEFAULT_DIGEST_LEN'.-  -- The 'Digest' is wiped from memory as soon as the 'Digest' becomes unused.+  -> output -- ^ The @output@ type could be @'BIO.Digest' len@. finalize h0 = unsafeDupablePerformIO $ do   (dig, _ :: BIO.Hasher) <- BAS.copyRet h0 BIO.finalize   pure dig {-# NOINLINE finalize #-} --- | Finalize incremental hashing and obtain a 'Digest' of length @len@ /after/--- the specified number of bytes of BLAKE3 output.+-- | Finalize incremental hashing and obtain the specified @len@gth of BLAKE3+-- output starting at the specified offset. -- -- @ -- 'finalize' h = 'finalizeSeek' h 0 -- @ finalizeSeek-  :: forall len-  .  KnownNat len+  :: forall len output+  .  BAS.ByteArrayN len output   => BIO.Hasher-  -> Word64     -- ^ Number of bytes to skip before obtaning the digest output.-  -> BIO.Digest len-  -- ^ Default digest length is 'BIO.DEFAULT_DIGEST_LEN'.-  -- The 'Digest' is wiped from memory as soon as the 'Digest' becomes unused.+  -> Word64     -- ^ BLAKE3 output offset.+  -> output     -- ^ The @output@ type could be @'BIO.Digest' len@. finalizeSeek h0 pos = unsafeDupablePerformIO $ do   (dig, _ :: BIO.Hasher) <- BAS.copyRet h0 $ \ph -> BIO.finalizeSeek ph pos   pure dig {-# NOINLINE finalizeSeek #-}-----------------------------------------------------------------------------------

@@ -17,15 +17,11 @@   , finalize   , finalizeSeek     -- * Digest-  , Digest-  , digest+  , Digest(..)     -- * Keyed hashing   , Key   , key-  , initKeyed     -- * Key derivation-  , Context-  , context   , initDerive     -- * Hasher   , Hasher@@ -42,19 +38,17 @@     -- * Low-level C bindings   , c_init   , c_init_keyed-  , c_init_derive_key+  , c_init_derive_key_raw   , c_update   , c_finalize   , c_finalize_seek   )   where -import Control.Monad (guard) import Data.Foldable import Data.Proxy import Data.String import Data.Word-import Foreign.C.String import Foreign.C.Types import Foreign.Marshal.Array (copyArray) import Foreign.Ptr@@ -64,53 +58,22 @@ import qualified Data.ByteArray as BA import qualified Data.ByteArray.Sized as BAS import qualified Data.ByteArray.Encoding as BA-import qualified Data.Memory.PtrMethods as BA  --------------------------------------------------------------------------------  -- | Output from BLAKE3 algorithm, of @len@ bytes. -- -- The default digest length for BLAKE3 is 'DEFAULT_DIGEST_LEN'.-data Digest (len :: Nat) where-  -- | We store things this way to avoid unnecessary conversions between-  -- different 'BA.ByteArrayAccess' when using 'digest' for reading a 'Digest'-  -- from a third party source.-  ---  -- Digest produced by this library are always allocated with 'BAS.allocRet'.-  Digest :: BA.ByteArrayAccess x => x -> Digest len---- | Obtain a digest containing bytes from a third-party source.------ This is useful if you want to use the 'Digest' datatype in your programs, but--- you are loading the pre-calculated digests from a database or similar.-digest-  :: forall len bin-  .  (KnownNat len, BA.ByteArrayAccess bin)-  => bin  -- ^ Raw digest bytes. Must have length @len@.-  -> Maybe (Digest len)  -- ^-digest bin-  | BA.length bin /= fromIntegral (natVal (Proxy @len)) = Nothing-  | otherwise = Just (Digest bin)---- | Constant time.-instance Eq (Digest len) where-  Digest a == Digest b = BA.constEq a b+newtype Digest (len :: Nat)+  = Digest (BAS.SizedByteArray len BA.ScrubbedBytes)+  deriving newtype ( Eq -- ^ Constant time.+                   , Ord+                   , BA.ByteArrayAccess+                   , BAS.ByteArrayN len )  -- | Base 16 (hexadecimal). instance Show (Digest len) where-  show (Digest x) = showBase16 x--instance BA.ByteArrayAccess (Digest len) where-  length (Digest x) = BA.length x-  withByteArray (Digest x) = BA.withByteArray x---- | Allocate a 'Digest'.--- The memory is wiped and freed as soon the 'Digest' becomes unused.-instance KnownNat len => BAS.ByteArrayN len (Digest len) where-  allocRet prx g = do-    let size = fromIntegral (natVal prx)-    (a, bs :: BA.ScrubbedBytes) <- BA.allocRet size g-    pure (a, Digest bs)+  show (Digest x) = showBase16 (BAS.unSizedByteArray x)  -- | When allocating a 'Digest', prefer to use 'BAS.alloc', which -- wipes and releases the memory as soon it becomes unused.@@ -135,7 +98,8 @@  -- | Constant time. instance Eq Key where-  Key a == Key b = BA.constEq a b+  (==) = BA.constEq+  {-# INLINE (==) #-}  -- | Base 16 (hexadecimal). instance Show Key where@@ -147,6 +111,7 @@   withByteArray (Key x) = BA.withByteArray x  -- | Allocate a 'Key'.+-- -- The memory is wiped and freed as soon as the 'Key' becomes unused. instance BAS.ByteArrayN KEY_LEN Key where   allocRet _ g = do@@ -173,70 +138,6 @@  -------------------------------------------------------------------------------- --- | Context for BLAKE3 key derivation. Obtain with 'context'.-newtype Context-  = Context BA.Bytes-  -- ^ NUL-terminated 'CString'. We store things this way so as to avoid-  -- re-creating the 'CString' each time we need to use this 'Context' in-  -- 'c_init_derive_key'. We never expose the NUL-terminating byte to users-  -- of this library.-  deriving newtype (Eq)---- We exclude the NUL-terminating byte. That's internal.-instance BA.ByteArrayAccess Context where-  length (Context x) = max 0 (BA.length x - 1)-  withByteArray c@(Context x) = BA.withByteArray (BA.takeView x (BA.length c))---- | Base 16 (hexadecimal).-instance Show Context where-  show = showBase16---- | 'fromString' is a /partial/ function that fails if the given 'String'--- contains 'Char's outside the range @['toEnum' 1 .. 'toEnum' 255]@.--- See 'context' for more details.-instance IsString Context where-  fromString s = case traverse charToWord8 s of-      Nothing -> error "Not a valid String for Context"-      Just w8s -> Context $! BA.pack (w8s <> [0])-    where-      charToWord8 :: Char -> Maybe Word8-      charToWord8 c = do-        let i = fromEnum c-        guard (i > 0 && i < 256)-        pure (fromIntegral i)---- | Obtain a 'Context' for BLAKE3 key derivation.------ The context should be hardcoded, globally unique, and--- application-specific.------ A good format for the context string is:------ @--- [application] [commit timestamp] [purpose]--- @------ For example:------ @--- example.com 2019-12-25 16:18:03 session tokens v1--- @-context-  :: BA.ByteArrayAccess bin-  => bin -- ^ If @bin@ contains null bytes, this function returns 'Nothing'.-  -> Maybe Context-context src-  | BA.any (0 ==) src = Nothing-  | otherwise = Just $ Context $-      let srcLen = BA.length src-          dstLen = srcLen + 1-      in BA.allocAndFreeze dstLen $ \pdst ->-         BA.withByteArray src $ \psrc -> do-           BA.memCopy pdst psrc srcLen-           pokeByteOff pdst srcLen (0 :: Word8)----------------------------------------------------------------------------------- showBase16 :: BA.ByteArrayAccess x => x -> String showBase16 = fmap (toEnum . fromIntegral)            . BA.unpack @BA.ScrubbedBytes@@ -246,16 +147,14 @@  -- | BLAKE3 hashing. hash-  :: forall len bin-  .  (KnownNat len, BA.ByteArrayAccess bin)-  => [bin]-  -- ^ Data to hash.-  -> IO (Digest len)-  -- ^ Default digest length is 'BIO.DEFAULT_DIGEST_LEN'.-  -- The 'Digest' is wiped from memory as soon as the 'Digest' becomes unused.-hash bins = do+  :: forall len digest bin+  .  (BAS.ByteArrayN len digest, BA.ByteArrayAccess bin)+  => Maybe Key  -- ^ Whether to use keyed hashing mode (for MAC, PRF).+  -> [bin]      -- ^ Data to hash.+  -> IO digest  -- ^ The @digest@ type could be @'Digest' len@.+hash yk bins = do   (dig, _ :: Hasher) <- BAS.allocRet Proxy $ \ph -> do-    init ph+    init ph yk     update ph bins     finalize ph   pure dig@@ -263,28 +162,24 @@ -- | Initialize a 'Hasher'. init   :: Ptr Hasher  -- ^ Obtain with 'BAS.alloc' or similar. It will be mutated.-  -> IO ()-init = c_init---- | Initialize a 'Hasher' in keyed mode.-initKeyed-  :: Ptr Hasher  -- ^ Obtain with 'BAS.alloc' or similar. It will be mutated.-  -> Key+  -> Maybe Key   -- ^ Whether to use keyed hashing mode (for MAC, PRF).   -> IO ()-initKeyed ph key0 =-  BA.withByteArray key0 $ \pkey ->-  c_init_keyed ph pkey+init ph Nothing     = c_init ph+init ph (Just key0) = BA.withByteArray key0 $ \pkey ->+                      c_init_keyed ph pkey  -- | Initialize a 'Hasher' in derivation mode. -- -- The input key material must be provided afterwards, using 'update'. initDerive-  :: Ptr Hasher  -- ^ Obtain with 'BAS.alloc' or similar. It will be mutated.-  -> Context+  :: forall context+  .  BA.ByteArrayAccess context+  => Ptr Hasher  -- ^ Obtain with 'BAS.alloc' or similar. It will be mutated.+  -> context   -> IO ()-initDerive ph (Context ctx) =+initDerive ph ctx =   BA.withByteArray ctx $ \pc ->-  c_init_derive_key ph pc+  c_init_derive_key_raw ph pc (fromIntegral (BA.length ctx))  -- | Update 'Hasher' state with new data. update@@ -298,32 +193,29 @@   BA.withByteArray bin $ \pbin ->   c_update ph pbin (fromIntegral (BA.length bin)) --- | Finalize incremental hashin and obtain a 'Digest'.+-- | Finalize incremental hashing and obtain a the BLAKE3 output of the+-- specified @len@gth. finalize-  :: forall len-  .  KnownNat len+  :: forall len output+  .  BAS.ByteArrayN len output   => Ptr Hasher -- ^ Obtain with 'modifyHasher'. It will be mutated.-  -> IO (Digest len)-  -- ^ Default digest length is 'BIO.DEFAULT_DIGEST_LEN'.-  -- The 'Digest' is wiped from memory as soon as the 'Digest' becomes unused.+  -> IO output  -- ^ The @output@ type could be @'Digest' len@. finalize ph =   BAS.alloc $ \pd ->   c_finalize ph pd (fromIntegral (natVal (Proxy @len))) --- | Finalize incremental hashing and obtain a 'Digest' of length @len@ /after/--- the specified number of bytes of BLAKE3 output.+-- | Finalize incremental hashing and obtain the specified @len@gth of BLAKE3+-- output starting at the specified offset. -- -- @ -- 'finalize' h = 'finalizeSeek' h 0 -- @ finalizeSeek-  :: forall len-  .  KnownNat len+  :: forall len output+  .  BAS.ByteArrayN len output   => Ptr Hasher -- ^ Obtain with 'modifyHasher'. It will be mutated.-  -> Word64     -- ^ Number of bytes to skip before obtaning the digest output.-  -> IO (Digest len)-  -- ^ Default digest length is 'BIO.DEFAULT_DIGEST_LEN'.-  -- The 'Digest' is wiped from memory as soon as the 'Digest' becomes unused.+  -> Word64     -- ^ BLAKE3 output offset.+  -> IO output finalizeSeek ph pos =   BAS.alloc $ \pd ->   c_finalize_seek ph pos pd (fromIntegral (natVal (Proxy @len)))@@ -361,16 +253,13 @@ -- Obtain with 'BLAKE3.hasher', 'BLAKE3.hasherKeyed'. newtype Hasher = Hasher (BAS.SizedByteArray HASHER_SIZE BA.ScrubbedBytes)   deriving newtype-    ( BA.ByteArrayAccess-      -- ^ Length is 'HASHER_SIZE'.+    ( Eq -- ^ Constant time.+    , BA.ByteArrayAccess -- ^ Length is 'HASHER_SIZE'.     , BAS.ByteArrayN HASHER_SIZE       -- ^ Allocate a 'Hasher'.       -- The memory is wiped and freed as soon as the 'Hasher' becomes unused.     ) -instance Eq Hasher where-  (==) = BA.eq- -- | Base 16 (hexadecimal). instance Show Hasher where   show = showBase16@@ -412,14 +301,16 @@                    -- Otherwise, any chunk of length 'KEY_LEN' will do.     -> IO () --- | @void blake3_hasher_init_derive_key(blake3_hasher *self, const char *context)@++-- | @void blake3_hasher_init_derive_key_raw(blake3_hasher *self, const void *context, size_t context_len)@ foreign import ccall unsafe-  "blake3.h blake3_hasher_init_derive_key"-  c_init_derive_key+  "blake3.h blake3_hasher_init_derive_key_raw"+  c_init_derive_key_raw     :: Ptr Hasher  -- ^ You can obtain with 'BAS.alloc'.                    -- Otherwise, any chunk of 'HASHER_SIZE' bytes aligned to                    -- 'HASHER_ALIGNMENT' will do.-    -> CString  -- ^ Context.+    -> Ptr Word8   -- ^ Context.+    -> CSize       -- ^ Context length.     -> IO ()  -- | @void blake3_hasher_update(blake3_hasher *self, const void *input, size_t input_len)@

@@ -29,7 +29,6 @@ tt_BLAKE3 = testGroup "BLAKE3"   [ tt_chunksOf   , tt_testVectors-  , tt_context   , tt_key   , tt_hasher   , tt_digest@@ -90,39 +89,39 @@ tt_hasher :: TestTree tt_hasher = testGroup "Hasher"   [ testCase "hasher: no-mutate" $ do-      let h1 = B.hasher-      "af1349b9f5f9a1a6a040" @=? show (B.finalize @10 h1)+      let h1 = B.init Nothing+      "af1349b9f5f9a1a6a040" @=? show (B.finalize h1 :: B.Digest 10)       let h2 = B.update @BA.ScrubbedBytes h1 ["upd"]-      "af1349b9f5f9a1a6a040" @=? show (B.finalize @10 h1)-      "3bad57477a5020f06859" @=? show (B.finalize @10 h2)+      "af1349b9f5f9a1a6a040" @=? show (B.finalize h1 :: B.Digest 10)+      "3bad57477a5020f06859" @=? show (B.finalize h2 :: B.Digest 10)       let h3 = B.update @BA.ScrubbedBytes h2 ["fin"]-      "af1349b9f5f9a1a6a040" @=? show (B.finalize @10 h1)-      "3bad57477a5020f06859" @=? show (B.finalize @10 h2)-      "48503741232720f6ca03" @=? show (B.finalize @10 h3)+      "af1349b9f5f9a1a6a040" @=? show (B.finalize h1 :: B.Digest 10)+      "3bad57477a5020f06859" @=? show (B.finalize h2 :: B.Digest 10)+      "48503741232720f6ca03" @=? show (B.finalize h3 :: B.Digest 10)    , testCase "hasherKeyed: no-mutate" $ do-      let h1 = B.hasherKeyed testVector_key-      "92b2b75604ed3c761f9d" @=? show (B.finalize @10 h1)+      let h1 = B.init (Just testVector_key)+      "92b2b75604ed3c761f9d" @=? show (B.finalize h1 :: B.Digest 10)       let h2 = B.update @BA.ScrubbedBytes h1 ["upd"]-      "92b2b75604ed3c761f9d" @=? show (B.finalize @10 h1)-      "f46255baa87a3280d644" @=? show (B.finalize @10 h2)+      "92b2b75604ed3c761f9d" @=? show (B.finalize h1 :: B.Digest 10)+      "f46255baa87a3280d644" @=? show (B.finalize h2 :: B.Digest 10)       let h3 = B.update @BA.ScrubbedBytes h2 ["fin"]-      "92b2b75604ed3c761f9d" @=? show (B.finalize @10 h1)-      "f46255baa87a3280d644" @=? show (B.finalize @10 h2)-      "b5ee60b5d89ac7e1289b" @=? show (B.finalize @10 h3)+      "92b2b75604ed3c761f9d" @=? show (B.finalize h1 :: B.Digest 10)+      "f46255baa87a3280d644" @=? show (B.finalize h2 :: B.Digest 10)+      "b5ee60b5d89ac7e1289b" @=? show (B.finalize h3 :: B.Digest 10)    , testCase "finalize vs finalizeSeek: zero" $ do-      let dig0 :: B.Digest 50 = B.finalize B.hasher-      let dig1 :: B.Digest 50 = B.finalizeSeek B.hasher 0+      let dig0 :: B.Digest 50 = B.finalize (B.init Nothing)+      let dig1 :: B.Digest 50 = B.finalizeSeek (B.init Nothing) 0       dig0 @=? dig1    , testCase "finalize vs finalizeSeek: non-zero" $ do-      let dig0 :: B.Digest 2050 = B.finalize B.hasher-      let dig1 :: B.Digest 50 = B.finalizeSeek B.hasher 2000+      let dig0 :: B.Digest 2050 = B.finalize (B.init Nothing)+      let dig1 :: B.Digest 50 = B.finalizeSeek (B.init Nothing) 2000       BA.drop 2000 (BA.convert dig0) @=? (BA.convert dig1 :: BA.Bytes)    , testCase "Storable" $ do-      let h1 = B.hasher+      let h1 = B.init Nothing       h2 <- BAS.alloc $ \ph2 -> poke ph2 h1       h1 @=? h2       h3 <- BA.withByteArray h1 peek@@ -132,31 +131,16 @@ tt_digest :: TestTree tt_digest = testGroup "Digest"   [ testCase "Storable" $ do-      let d1 :: B.Digest 400 = B.hash ([] :: [BA.Bytes])+      let d1 :: B.Digest 400 = B.hash Nothing ([] :: [BA.Bytes])       d2 <- BAS.alloc $ \pd2 -> poke pd2 d1       d1 @=? d2       d3 <- BA.withByteArray d1 peek       d1 @=? d3   ] -testVector_context :: B.Context+testVector_context :: BA.ScrubbedBytes testVector_context = "BLAKE3 2019-12-27 16:29:52 test vectors context" -tt_context :: TestTree-tt_context = testGroup "Context"-  [ testCase "empty" $ do-      Just "" @=? fmap show (B.context ("" :: BA.ScrubbedBytes))-  , testCase "non-empty" $ do-      Just "6869" @=? fmap show (B.context ("hi" :: BA.ScrubbedBytes))-  , testCase "has null" $ do-      Nothing @=? B.context ("\NUL" :: BA.ScrubbedBytes)-      Nothing @=? B.context ("hi\NUL" :: BA.ScrubbedBytes)-  , testCase "fromString" $ do-      Just "" @=? B.context ("" :: BA.ScrubbedBytes)-      Just "a" @=? B.context ("a" :: BA.ScrubbedBytes)-      Just "ab" @=? B.context ("ab" :: BA.ScrubbedBytes)-  ]- tt_testVectors :: TestTree tt_testVectors = testGroup "TestVectors" (fmap mkTestTree testVectors) @@ -166,7 +150,7 @@   in BA.pack <$> chunksOf 100 w8s  testVector_key :: B.Key-Just testVector_key = B.key+testVector_key = maybe undefined id $ B.key   ("whats the Elvish word for friend" :: BA.ScrubbedBytes)  tt_key :: TestTree@@ -202,10 +186,10 @@   in case someNatVal (fromIntegral digestLen) of        Nothing -> error "Bad tv_inputLen"        Just (SomeNat (Proxy :: Proxy len)) ->-         let digest :: B.Digest len = B.hash input-             digestF :: B.Digest len = hashF input-             digestKeyed :: B.Digest len = B.hashKeyed testVector_key input-             digestKeyedF :: B.Digest len = hashKeyedF testVector_key input+         let digest :: B.Digest len = B.hash Nothing input+             digestF :: B.Digest len = hashF Nothing input+             digestKeyed :: B.Digest len = B.hash (Just testVector_key) input+             digestKeyedF :: B.Digest len = hashF (Just testVector_key) input              digestDerived :: B.Digest len = B.derive testVector_context input          in [ testCase "hash" $ tv_hash tv @=? show digest             , testCase "hashF" $ tv_hash tv @=? show digestF@@ -214,13 +198,9 @@             , testCase "derivedKey" $ tv_derivedKey tv @=? show digestDerived             ] --- | Same as 'B.hashKeyed'. For testing.-hashKeyedF :: (KnownNat len, BA.ByteArrayAccess bin) => B.Key -> [bin] -> B.Digest len-hashKeyedF k = B.finalize . B.update (B.hasherKeyed k)- -- | Same as 'B.hash'. For testing.-hashF :: (KnownNat len, BA.ByteArrayAccess bin) => [bin] -> B.Digest len-hashF = B.finalize . B.update B.hasher+hashF :: (KnownNat len, BA.ByteArrayAccess bin) => Maybe B.Key -> [bin] -> B.Digest len+hashF yk = B.finalize . B.update (B.init yk)  testVectors :: [TestVector] testVectors = [