souffle-haskell 1.0.0 → 1.1.0

raw patch · 32 files changed

+1944/−372 lines, 32 filesdep +arraydep +hedgehogdep +hspec-hedgehogPVP ok

version bump matches the API change (PVP)

Dependencies added: array, hedgehog, hspec-hedgehog

API changes (from Hackage documentation)

Files

• CHANGELOG.md +28/−7
• README.md +10/−12
• cbits/souffle.cpp +105/−56
• cbits/souffle.h +66/−23
• cbits/souffle/CompiledSouffle.h +0/−1
• cbits/souffle/CompiledTuple.h +1/−1
• cbits/souffle/IOSystem.h +6/−0
• cbits/souffle/IterUtils.h +0/−137
• cbits/souffle/ReadStreamCSV.h +12/−1
• cbits/souffle/ReadStreamJSON.h +368/−0
• cbits/souffle/ReadStreamSQLite.h +23/−3
• cbits/souffle/SerialisationStream.h +5/−9
• cbits/souffle/SouffleInterface.h +21/−20
• cbits/souffle/WriteStreamCSV.h +36/−2
• cbits/souffle/WriteStreamJSON.h +298/−0
• cbits/souffle/WriteStreamSQLite.h +22/−3
• cbits/souffle/utility/ContainerUtil.h +113/−0
• cbits/souffle/utility/EvaluatorUtil.h +7/−17
• cbits/souffle/utility/FileUtil.h +8/−4
• cbits/souffle/utility/MiscUtil.h +0/−1
• lib/Language/Souffle.hs +0/−7
• lib/Language/Souffle/Compiled.hs +49/−7
• lib/Language/Souffle/Internal.hs +40/−10
• lib/Language/Souffle/Internal/Bindings.hs +62/−6
• lib/Language/Souffle/Internal/Constraints.hs +4/−1
• lib/Language/Souffle/Interpreted.hs +47/−18
• lib/Language/Souffle/Marshal.hs +27/−6
• souffle-haskell.cabal +22/−13
• tests/Test/Language/Souffle/CompiledSpec.hs +12/−1
• tests/Test/Language/Souffle/InterpretedSpec.hs +12/−0
• tests/Test/Language/Souffle/MarshalSpec.hs +141/−6
• tests/fixtures/round_trip.cpp +399/−0

@@ -1,10 +1,31 @@- # Changelog  All notable changes to this project (as seen by library users) will be documented in this file. The CHANGELOG is available on [Github](https://github.com/luc-tielen/souffle-haskell.git/CHANGELOG.md). +## [1.1.0] - 2020-07-26++### Added++- getFacts can now return facts in a fixed size array (from the array package).+- Added support for facts containing "unsigned" values.+- Added support for facts containing "float" values.++### Fixed++- The `run` function in `Language.Souffle.Interpeted` now always closes+  stdout and stderr handles of the external souffle process.++### Removed++- `Language.Souffle` module is removed since it only existed due to legacy+  reasons. This removal forces users to be explicit about the mode they are+  using souffle-haskell in (interpreted or compiled mode). If you experience+  compilation errors, rename all imports of `Language.Souffle` to+  `Language.Souffle.Compiled`.+ ## [1.0.0] - 2020-07-09+ ### Changed  - Libraries using souffle-haskell are now "self-contained": if a project@@ -20,27 +41,27 @@   correctly integrated by adding the files to `cxx-sources`   in package.yaml / cabal file. - ## [0.2.3] - 2020-05-21+ ### Changed  - Optimize performance when marshalling and unmarshalling facts. - ## [0.2.2] - 2020-04-30+ ### Changed  - Fix compile time issue when generically deriving `Marshal` typeclass   for data types with more than 3 fields. - ## [0.2.1] - 2020-04-25+ ### Changed  - Trimmed dependencies to make the library more lightweight. - ## [0.2.0] - 2020-04-22+ ### Added  - Added Language.Souffle.Interpreted module for running Souffle programs in interpreted mode.@@ -54,8 +75,8 @@ - Introduced Language.Souffle.Class module as separation of the typeclass and the   Language.Souffle.Compiled module to offer a uniform API in both interpreted and compiled mode. - ## [0.1.0] - 2019-12-21+ ### Added  - Added Marshal instance for lazy and strict Text@@ -66,8 +87,8 @@   This allows for a more efficient representation in memory as well   as being able to allocate all needed memory once before collecting facts. - ## [0.0.1] - 2019-10-23+ ### Added  - Initial version of the library

@@ -1,4 +1,3 @@- # Souffle-haskell  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/luc-tielen/souffle-haskell/blob/master/LICENSE)@@ -11,7 +10,6 @@ Fun fact: this library combines both functional programming (Haskell), logic programming (Datalog / Souffle) and imperative / OO programming (C / C++). - ## Motivating example  Let's first write a datalog program that can check if one point@@ -60,7 +58,8 @@ data Path = Path  -- Facts are represented in Haskell as simple product types,--- Numbers map to Int32, symbols to Strings / Text.+-- Numbers map to Int32, unsigned to Word32, floats to Float,+-- symbols to Strings / Text.  data Edge = Edge String String   deriving (Eq, Show, Generic)@@ -121,7 +120,6 @@ For more examples of how to use the top level API, you can also take a look at the tests. - ## Getting started  This library assumes that the Souffle include paths are properly set.@@ -141,7 +139,7 @@ cxx-options:   - -D__EMBEDDED_SOUFFLE__ cxx-sources:-  - /path/to/FILE.cpp  # be sure to change this according to what you need!+  - /path/to/FILE.cpp # be sure to change this according to what you need!  # ... ```@@ -161,6 +159,13 @@   souffle-haskell: -optcxx-std=c++17 ``` +## Documentation++The documentation for the library can be found on+[Hackage](https://hackage.haskell.org/package/souffle-haskell).+`Language.Souffle.Class` is a good starting point for getting an overview of+the top level API.+ ## Supported modes  Souffle programs can be run in 2 ways. They can either run in **interpreted** mode@@ -169,7 +174,6 @@ modes (since version 0.2.0). The two variants have only a few minor differences and can be swapped fairly easily. - ### Interpreted mode  This is probably the mode you want to start out with if you are developing a@@ -185,7 +189,6 @@    functionality. This will clean up the generated CSV fact files located in    a temporary directory. - #### Interpreter configuration  The interpreter uses CSV files to read or write facts. The configuration@@ -205,7 +208,6 @@ The separators in the CSV fact files cannot be configured at the moment. A tab character (`'\t'`) is used to separate the different columns. - ### Compiled mode  Once the prototyping phase of the Datalog algorithm is over, it is advised@@ -221,12 +223,10 @@  The [motivating example](#motivating-example) is a complete example for the compiled mode. - ## Contributing  TLDR: Nix-based project; the Makefile contains the most commonly used commands. - Long version:  The project makes use of [Nix](https://nixos.org/nix/download.html) to setup the development environment.@@ -245,9 +245,7 @@ $ make hoogle     # starts a local hoogle webserver ``` - ## Issues  Found an issue or missing a piece of functionality? Please open an issue with a description of the problem.-

@@ -1,147 +1,196 @@ #include "souffle/SouffleInterface.h" #include "souffle.h" +template <typename T>+void tuple_push_value(tuple_t *tuple, const T &value)+{+    auto t = reinterpret_cast<souffle::tuple *>(tuple);+    assert(t);+    *t << value;+} -extern "C" {-    souffle_t* souffle_init(const char* progName) {+template <typename T>+void tuple_pop_value(tuple_t *tuple, T *result)+{+    auto t = reinterpret_cast<souffle::tuple *>(tuple);+    assert(t);+    assert(result);+    *t >> *result;+}++extern "C"+{+    souffle_t *souffle_init(const char *progName)+    {         auto prog = souffle::ProgramFactory::newInstance(progName);-        return reinterpret_cast<souffle_t*>(prog);+        return reinterpret_cast<souffle_t *>(prog);     } -    void souffle_free(souffle_t* program) {-        auto prog = reinterpret_cast<souffle::SouffleProgram*>(program);+    void souffle_free(souffle_t *program)+    {+        auto prog = reinterpret_cast<souffle::SouffleProgram *>(program);         assert(prog);         delete prog;     } -    void souffle_set_num_threads(souffle_t* program, size_t num_cores) {-        auto prog = reinterpret_cast<souffle::SouffleProgram*>(program);+    void souffle_set_num_threads(souffle_t *program, size_t num_cores)+    {+        auto prog = reinterpret_cast<souffle::SouffleProgram *>(program);         assert(prog);         prog->setNumThreads(num_cores);     } -    size_t souffle_get_num_threads(souffle_t* program) {-        auto prog = reinterpret_cast<souffle::SouffleProgram*>(program);+    size_t souffle_get_num_threads(souffle_t *program)+    {+        auto prog = reinterpret_cast<souffle::SouffleProgram *>(program);         assert(prog);         return prog->getNumThreads();     } -    void souffle_run(souffle_t* program) {-        auto prog = reinterpret_cast<souffle::SouffleProgram*>(program);+    void souffle_run(souffle_t *program)+    {+        auto prog = reinterpret_cast<souffle::SouffleProgram *>(program);         assert(prog);         prog->run();     } -    void souffle_load_all(souffle_t* program, const char* input_directory) {-        auto prog = reinterpret_cast<souffle::SouffleProgram*>(program);+    void souffle_load_all(souffle_t *program, const char *input_directory)+    {+        auto prog = reinterpret_cast<souffle::SouffleProgram *>(program);         assert(prog);         assert(input_directory);         prog->loadAll(input_directory);     } -    void souffle_print_all(souffle_t* program, const char* output_directory) {-        auto prog = reinterpret_cast<souffle::SouffleProgram*>(program);+    void souffle_print_all(souffle_t *program, const char *output_directory)+    {+        auto prog = reinterpret_cast<souffle::SouffleProgram *>(program);         assert(prog);         assert(output_directory);         prog->printAll(output_directory);     } -    relation_t* souffle_relation(souffle_t* program, const char* relation_name) {-        auto prog = reinterpret_cast<souffle::SouffleProgram*>(program);+    relation_t *souffle_relation(souffle_t *program, const char *relation_name)+    {+        auto prog = reinterpret_cast<souffle::SouffleProgram *>(program);         assert(prog);         assert(relation_name);         auto relation = prog->getRelation(relation_name);         assert(relation);-        return reinterpret_cast<relation_t*>(relation);+        return reinterpret_cast<relation_t *>(relation);     } -    size_t souffle_relation_tuple_count(relation_t* relation) {-        auto rel = reinterpret_cast<souffle::Relation*>(relation);+    size_t souffle_relation_tuple_count(relation_t *relation)+    {+        auto rel = reinterpret_cast<souffle::Relation *>(relation);         assert(rel);         return rel->size();     } -    struct relation_iterator {+    struct relation_iterator+    {         using iterator_t = souffle::Relation::iterator;         iterator_t iterator; -        relation_iterator(const iterator_t& it)+        relation_iterator(const iterator_t &it)             : iterator(it) {}     }; -    relation_iterator_t* souffle_relation_iterator(relation_t* relation) {-        auto rel = reinterpret_cast<souffle::Relation*>(relation);+    relation_iterator_t *souffle_relation_iterator(relation_t *relation)+    {+        auto rel = reinterpret_cast<souffle::Relation *>(relation);         assert(rel);-        relation_iterator_t* it = new relation_iterator_t(rel->begin());+        relation_iterator_t *it = new relation_iterator_t(rel->begin());         return it;     } -    void souffle_relation_iterator_free(relation_iterator_t* iterator) {+    void souffle_relation_iterator_free(relation_iterator_t *iterator)+    {         assert(iterator);         delete iterator;     } -    tuple_t* souffle_relation_iterator_next(relation_iterator_t* iterator) {+    tuple_t *souffle_relation_iterator_next(relation_iterator_t *iterator)+    {         assert(iterator);-        auto tuple = reinterpret_cast<tuple_t*>(&*iterator->iterator);+        auto tuple = reinterpret_cast<tuple_t *>(&*iterator->iterator);         ++iterator->iterator;         return tuple;     } -    bool souffle_contains_tuple(relation_t* relation, tuple_t* tuple) {-        auto rel = reinterpret_cast<souffle::Relation*>(relation);-        auto t = reinterpret_cast<souffle::tuple*>(tuple);+    bool souffle_contains_tuple(relation_t *relation, tuple_t *tuple)+    {+        auto rel = reinterpret_cast<souffle::Relation *>(relation);+        auto t = reinterpret_cast<souffle::tuple *>(tuple);         assert(rel);         assert(t);         return rel->contains(*t);     } -    tuple_t *souffle_tuple_alloc(relation_t* relation) {-        auto rel = reinterpret_cast<souffle::Relation*>(relation);+    tuple_t *souffle_tuple_alloc(relation_t *relation)+    {+        auto rel = reinterpret_cast<souffle::Relation *>(relation);         assert(rel);         auto tuple = new souffle::tuple(rel);-        return reinterpret_cast<tuple_t*>(tuple);+        return reinterpret_cast<tuple_t *>(tuple);     } -    void souffle_tuple_free(tuple_t* tuple) {-        auto t = reinterpret_cast<souffle::tuple*>(tuple);+    void souffle_tuple_free(tuple_t *tuple)+    {+        auto t = reinterpret_cast<souffle::tuple *>(tuple);         assert(t);         delete t;     } -    void souffle_tuple_push_int(tuple_t* tuple, int32_t value) {-        auto t = reinterpret_cast<souffle::tuple*>(tuple);-        assert(t);-        *t << value;+    void souffle_tuple_push_int32(tuple_t *tuple, int32_t value)+    {+        tuple_push_value(tuple, value);     } -    void souffle_tuple_push_string(tuple_t* tuple, const char* value) {-        auto t = reinterpret_cast<souffle::tuple*>(tuple);-        assert(t);-        *t << value;+    void souffle_tuple_push_uint32(tuple_t *tuple, uint32_t value)+    {+        tuple_push_value(tuple, value);     } -    void souffle_tuple_add(relation_t* relation, tuple_t* tuple) {-        auto rel = reinterpret_cast<souffle::Relation*>(relation);-        auto t = reinterpret_cast<souffle::tuple*>(tuple);+    void souffle_tuple_push_float(tuple_t *tuple, float value)+    {+        tuple_push_value(tuple, value);+    }++    void souffle_tuple_push_string(tuple_t *tuple, const char *value)+    {+        tuple_push_value(tuple, value);+    }++    void souffle_tuple_add(relation_t *relation, tuple_t *tuple)+    {+        auto rel = reinterpret_cast<souffle::Relation *>(relation);+        auto t = reinterpret_cast<souffle::tuple *>(tuple);         assert(rel);         assert(t);         rel->insert(*t);     } -    void souffle_tuple_pop_int(tuple_t* tuple, int32_t* result) {-        auto t = reinterpret_cast<souffle::tuple*>(tuple);-        assert(t);-        assert(result);-        *t >> *result;+    void souffle_tuple_pop_int32(tuple_t *tuple, int32_t *result)+    {+        tuple_pop_value(tuple, result);     } -    void souffle_tuple_pop_string(tuple_t* tuple, char** result) {-        auto t = reinterpret_cast<souffle::tuple*>(tuple);-        assert(t);+    void souffle_tuple_pop_uint32(tuple_t *tuple, uint32_t *result)+    {+        tuple_pop_value(tuple, result);+    }++    void souffle_tuple_pop_float(tuple_t *tuple, float *result)+    {+        tuple_pop_value(tuple, result);+    }++    void souffle_tuple_pop_string(tuple_t *tuple, char **result)+    {         assert(result);         std::string value;-        *t >> value;+        tuple_pop_value(tuple, &value);         *result = strdup(value.c_str());     } }

@@ -5,7 +5,8 @@ #include <stdbool.h>  #ifdef __cplusplus-extern "C" {+extern "C"+{ #endif      // Opaque struct representing a Souffle program@@ -25,21 +26,21 @@      * If a valid pointer is returned, it needs to be freed by "souffle_free"      * after it is no longer needed.      */-    souffle_t* souffle_init(const char* progName);+    souffle_t *souffle_init(const char *progName);      /*      * Frees the memory in use by "program".      * You need to check if the pointer is non-NULL before passing it to this      * function. Not doing so results in undefined behavior.      */-    void souffle_free(souffle_t* program);+    void souffle_free(souffle_t *program);      /*      * Sets the number of cores this program should use.      * You need to check if the passed pointer is non-NULL before passing it      * to this function. Not doing so results in undefined behavior.      */-    void souffle_set_num_threads(souffle_t* program, size_t num_cores);+    void souffle_set_num_threads(souffle_t *program, size_t num_cores);      /*      * Gets the number of cores this program should use.@@ -47,21 +48,21 @@      * to this function. Not doing so results in undefined behavior.      * Returns the number of cores the program will use.      */-    size_t souffle_get_num_threads(souffle_t* program);+    size_t souffle_get_num_threads(souffle_t *program);      /*      * Runs the Souffle program.      * You need to check if the pointer is non-NULL before passing it to this      * function. Not doing so results in undefined behavior.      */-    void souffle_run(souffle_t* program);+    void souffle_run(souffle_t *program);      /*      * Load all facts from files in a certain directory.      * You need to check if both pointers are non-NULL before passing it to this      * function. Not doing so results in undefined behavior.      */-    void souffle_load_all(souffle_t* program, const char* input_directory);+    void souffle_load_all(souffle_t *program, const char *input_directory);      /*      * Write out all facts of the program to CSV files@@ -70,7 +71,7 @@      * You need to check if the pointer is non-NULL before passing it to this      * function. Not doing so results in undefined behavior.      */-    void souffle_print_all(souffle_t* program, const char* output_directory);+    void souffle_print_all(souffle_t *program, const char *output_directory);      /*      * Lookup a relation in the Souffle program.@@ -80,7 +81,7 @@      * The returned pointer can be NULL if the relation is not found.      * The pointer does not need to be freed, it is managed by the Souffle program.      */-    relation_t* souffle_relation(souffle_t* program, const char* relation_name);+    relation_t *souffle_relation(souffle_t *program, const char *relation_name);      /*      * Gets the amount of tuples found in a relation.@@ -89,7 +90,7 @@      *      * Returns the amount of tuples found in a relation.      */-    size_t souffle_relation_tuple_count(relation_t* relation);+    size_t souffle_relation_tuple_count(relation_t *relation);      /*      * Create an iterator for iterating over the facts of a relation.@@ -99,14 +100,14 @@      * The returned pointer needs to be freed up with      * "souffle_relation_iterator_free" after it is no longer needed.      */-    relation_iterator_t* souffle_relation_iterator(relation_t* relation);+    relation_iterator_t *souffle_relation_iterator(relation_t *relation);      /*      * Frees a relation_iterator pointer.      * You need to check if the passed pointer is non-NULL before passing it      * to this function. Not doing so results in undefined behavior.      */-    void souffle_relation_iterator_free(relation_iterator_t* iterator);+    void souffle_relation_iterator_free(relation_iterator_t *iterator);      /*      * Advances the relation iterator by 1 position.@@ -117,7 +118,7 @@      *      * Returns a pointer to the next record. This pointer is not allowed to be freed.      */-    tuple_t* souffle_relation_iterator_next(relation_iterator_t* iterator);+    tuple_t *souffle_relation_iterator_next(relation_iterator_t *iterator);      /*      * Checks if a relation contains a certain tuple.@@ -126,7 +127,7 @@      *      * Returns true if the tuple was found in the relation; otherwise false.      */-    bool souffle_contains_tuple(relation_t* relation, tuple_t* tuple);+    bool souffle_contains_tuple(relation_t *relation, tuple_t *tuple);      /*      * Allocates memory for a tuple to be added to a relation.@@ -136,33 +137,53 @@      * Returns a pointer to a new tuple. Use "souffle_tuple_free" when tuple      * is no longer required.      */-    tuple_t* souffle_tuple_alloc(relation_t* relation);+    tuple_t *souffle_tuple_alloc(relation_t *relation);      /*      * Frees memory of a tuple that was previously allocated.      * You need to check if the passed pointer is non-NULL before passing it      * to this function. Not doing so results in undefined behavior.      */-    void souffle_tuple_free(tuple_t* tuple);+    void souffle_tuple_free(tuple_t *tuple);      /*      * Adds a tuple to a relation.      * You need to check if both passed pointers are non-NULL before passing it      * to this function. Not doing so results in undefined behavior.      */-    void souffle_tuple_add(relation_t* relation, tuple_t* tuple);+    void souffle_tuple_add(relation_t *relation, tuple_t *tuple);      /*-     * Pushes an integer value into a tuple.+     * Pushes a 32 bit signed integer value into a tuple.      * You need to check if the passed pointer is non-NULL before passing it      * to this function. Not doing so results in undefined behavior.      *      * Pushing an integer value onto a tuple that expects another type results      * in a crash.      */-    void souffle_tuple_push_int(tuple_t* tuple, int32_t value);+    void souffle_tuple_push_int32(tuple_t *tuple, int32_t value);      /*+     * Pushes a 32 bit unsigned integer value into a tuple.+     * You need to check if the passed pointer is non-NULL before passing it+     * to this function. Not doing so results in undefined behavior.+     *+     * Pushing an integer value onto a tuple that expects another type results+     * in a crash.+     */+    void souffle_tuple_push_uint32(tuple_t *tuple, uint32_t value);++    /*+     * Pushes a float value into a tuple.+     * You need to check if the passed pointer is non-NULL before passing it+     * to this function. Not doing so results in undefined behavior.+     *+     * Pushing a float value onto a tuple that expects another type results+     * in a crash.+     */+    void souffle_tuple_push_float(tuple_t *tuple, float value);++    /*      * Pushes a string value into a tuple.      * You need to check if the passed pointer is non-NULL before passing it      * to this function. Not doing so results in undefined behavior.@@ -170,10 +191,10 @@      * Pushing a string value onto a tuple that expects another type results      * in a crash.      */-    void souffle_tuple_push_string(tuple_t* tuple, const char* value);+    void souffle_tuple_push_string(tuple_t *tuple, const char *value);      /*-     * Extracts an integer value from a tuple.+     * Extracts a 32 bit signed integer value from a tuple.      * You need to check if the passed pointer is non-NULL before passing it      * to this function. Not doing so results in undefined behavior.      *@@ -181,9 +202,31 @@      * in a crash.      * The popped integer will be stored in the result pointer.      */-    void souffle_tuple_pop_int(tuple_t* tuple, int32_t* result);+    void souffle_tuple_pop_int32(tuple_t *tuple, int32_t *result);      /*+     * Extracts a 32 bit unsigned integer value from a tuple.+     * You need to check if the passed pointer is non-NULL before passing it+     * to this function. Not doing so results in undefined behavior.+     *+     * Extracting an integer value from a tuple that expects another type results+     * in a crash.+     * The popped integer will be stored in the result pointer.+     */+    void souffle_tuple_pop_uint32(tuple_t *tuple, uint32_t *result);++    /*+     * Extracts a float value from a tuple.+     * You need to check if the passed pointer is non-NULL before passing it+     * to this function. Not doing so results in undefined behavior.+     *+     * Extracting a float value from a tuple that expects another type results+     * in a crash.+     * The popped integer will be stored in the result pointer.+     */+    void souffle_tuple_pop_float(tuple_t *tuple, float *result);++    /*      * Extracts a string value from a tuple.      * You need to check if the passed pointer is non-NULL before passing it      * to this function. Not doing so results in undefined behavior.@@ -192,7 +235,7 @@      * in a crash.      * The popped string will be stored in the result pointer.      */-    void souffle_tuple_pop_string(tuple_t* tuple, char** result);+    void souffle_tuple_pop_string(tuple_t *tuple, char **result);  #ifdef __cplusplus }

@@ -20,7 +20,6 @@ #include "souffle/CompiledTuple.h" #include "souffle/EquivalenceRelation.h" #include "souffle/IOSystem.h"-#include "souffle/IterUtils.h" #include "souffle/RamTypes.h" #include "souffle/RecordTable.h" #include "souffle/SignalHandler.h"

@@ -40,7 +40,7 @@     // the stored data     Domain data[arity]; -    // constructores, destructors and assignment are default+    // constructors, destructors and assignment are default      // provide access to components     const Domain& operator[](std::size_t index) const {

@@ -17,9 +17,11 @@ #include "RamTypes.h" #include "ReadStream.h" #include "ReadStreamCSV.h"+#include "ReadStreamJSON.h" #include "SymbolTable.h" #include "WriteStream.h" #include "WriteStreamCSV.h"+#include "WriteStreamJSON.h"  #ifdef USE_SQLITE #include "ReadStreamSQLite.h"@@ -77,9 +79,13 @@     IOSystem() {         registerReadStreamFactory(std::make_shared<ReadFileCSVFactory>());         registerReadStreamFactory(std::make_shared<ReadCinCSVFactory>());+        registerReadStreamFactory(std::make_shared<ReadFileJSONFactory>());+        registerReadStreamFactory(std::make_shared<ReadCinJSONFactory>());         registerWriteStreamFactory(std::make_shared<WriteFileCSVFactory>());         registerWriteStreamFactory(std::make_shared<WriteCoutCSVFactory>());         registerWriteStreamFactory(std::make_shared<WriteCoutPrintSizeFactory>());+        registerWriteStreamFactory(std::make_shared<WriteFileJSONFactory>());+        registerWriteStreamFactory(std::make_shared<WriteCoutJSONFactory>()); #ifdef USE_SQLITE         registerReadStreamFactory(std::make_shared<ReadSQLiteFactory>());         registerWriteStreamFactory(std::make_shared<WriteSQLiteFactory>());

@@ -1,137 +0,0 @@-/*- * Souffle - A Datalog Compiler- * Copyright (c) 2013, 2015, Oracle and/or its affiliates. All rights reserved- * Licensed under the Universal Permissive License v 1.0 as shown at:- * - https://opensource.org/licenses/UPL- * - <souffle root>/licenses/SOUFFLE-UPL.txt- */--/************************************************************************- *- * @file IterUtils.h- *- * A (growing) collection of generic iterator utilities.- *- ***********************************************************************/--#pragma once--#include <iterator>-#include <type_traits>--namespace souffle {--/**- * A wrapper for an iterator obtaining pointers of a certain type,- * dereferencing values before forwarding them to the consumer.- *- * @tparam Iter ... the type of wrapped iterator- * @tparam T    ... the value to be accessed by the resulting iterator- */-template <typename Iter, typename T = typename std::remove_pointer<typename Iter::value_type>::type>-struct IterDerefWrapper : public std::iterator<std::forward_iterator_tag, T> {-    /* The nested iterator. */-    Iter iter;--public:-    // some constructores-    IterDerefWrapper() = default;-    IterDerefWrapper(const Iter& iter) : iter(iter) {}--    // defaulted copy and move constructors-    IterDerefWrapper(const IterDerefWrapper&) = default;-    IterDerefWrapper(IterDerefWrapper&&) = default;--    // default assignment operators-    IterDerefWrapper& operator=(const IterDerefWrapper&) = default;-    IterDerefWrapper& operator=(IterDerefWrapper&&) = default;--    /* The equality operator as required by the iterator concept. */-    bool operator==(const IterDerefWrapper& other) const {-        return iter == other.iter;-    }--    /* The not-equality operator as required by the iterator concept. */-    bool operator!=(const IterDerefWrapper& other) const {-        return iter != other.iter;-    }--    /* The deref operator as required by the iterator concept. */-    const T& operator*() const {-        return **iter;-    }--    /* Support for the pointer operator. */-    const T* operator->() const {-        return &(**iter);-    }--    /* The increment operator as required by the iterator concept. */-    IterDerefWrapper& operator++() {-        ++iter;-        return *this;-    }-};--/**- * A factory function enabling the construction of a dereferencing- * iterator utilizing the automated deduction of template parameters.- */-template <typename Iter>-IterDerefWrapper<Iter> derefIter(const Iter& iter) {-    return IterDerefWrapper<Iter>(iter);-}--// ----------------------------------------------------------------------//                        Single-Value-Iterator-// -----------------------------------------------------------------------/**- * An iterator to be utilized if there is only a single element to iterate over.- */-template <typename T>-class SingleValueIterator : public std::iterator<std::forward_iterator_tag, T> {-    T value;--    bool end = true;--public:-    SingleValueIterator() = default;--    SingleValueIterator(const T& value) : value(value), end(false) {}--    // a copy constructor-    SingleValueIterator(const SingleValueIterator& other) = default;--    // an assignment operator-    SingleValueIterator& operator=(const SingleValueIterator& other) = default;--    // the equality operator as required by the iterator concept-    bool operator==(const SingleValueIterator& other) const {-        // only equivalent if pointing to the end-        return end && other.end;-    }--    // the not-equality operator as required by the iterator concept-    bool operator!=(const SingleValueIterator& other) const {-        return !(*this == other);-    }--    // the deref operator as required by the iterator concept-    const T& operator*() const {-        return value;-    }--    // support for the pointer operator-    const T* operator->() const {-        return &value;-    }--    // the increment operator as required by the iterator concept-    SingleValueIterator& operator++() {-        end = true;-        return *this;-    }-};--}  // end of namespace souffle

@@ -273,8 +273,19 @@     ~ReadFileCSV() override = default;  protected:+    /**+     * Return given filename or construct from relation name.+     * Default name is [configured path]/[relation name].facts+     *+     * @param rwOperation map of IO configuration options+     * @return input filename+     */     static std::string getFileName(const std::map<std::string, std::string>& rwOperation) {-        return getOr(rwOperation, "filename", rwOperation.at("name") + ".facts");+        auto name = getOr(rwOperation, "filename", rwOperation.at("name") + ".facts");+        if (name.front() != '/') {+            name = getOr(rwOperation, "fact-dir", ".") + "/" + name;+        }+        return name;     }      std::string baseName;

@@ -0,0 +1,368 @@+/*+ * Souffle - A Datalog Compiler+ * Copyright (c) 2020, The Souffle Developers. All rights reserved+ * Licensed under the Universal Permissive License v 1.0 as shown at:+ * - https://opensource.org/licenses/UPL+ * - <souffle root>/licenses/SOUFFLE-UPL.txt+ */++/************************************************************************+ *+ * @file ReadStreamJSON.h+ *+ ***********************************************************************/++#pragma once++#include "RamTypes.h"+#include "ReadStream.h"+#include "SymbolTable.h"+#include "utility/ContainerUtil.h"+#include "utility/FileUtil.h"+#include "utility/StringUtil.h"++#include <algorithm>+#include <cassert>+#include <cstddef>+#include <cstdint>+#include <fstream>+#include <iostream>+#include <map>+#include <memory>+#include <queue>+#include <sstream>+#include <stdexcept>+#include <string>+#include <tuple>+#include <vector>++namespace souffle {+class RecordTable;++class ReadStreamJSON : public ReadStream {+public:+    ReadStreamJSON(std::istream& file, const std::map<std::string, std::string>& rwOperation,+            SymbolTable& symbolTable, RecordTable& recordTable)+            : ReadStream(rwOperation, symbolTable, recordTable), file(file), pos(0), isInitialized(false) {+        std::string err;+        params = Json::parse(rwOperation.at("params"), err);+        if (err.length() > 0) {+            fatal("cannot get internal params: %s", err);+        }+    }++protected:+    std::istream& file;+    size_t pos;+    Json jsonSource;+    Json params;+    bool isInitialized;+    bool useObjects;+    std::map<const std::string, const size_t> paramIndex;++    std::unique_ptr<RamDomain[]> readNextTuple() override {+        // for some reasons we cannot initalized our json objects in constructor+        // otherwise it will segfault, so we initialize in the first call+        if (!isInitialized) {+            isInitialized = true;+            std::string error = "";+            std::string source(std::istreambuf_iterator<char>(file), {});++            jsonSource = Json::parse(source, error);+            // it should be wrapped by an extra array+            if (error.length() > 0 || !jsonSource.is_array()) {+                fatal("cannot deserialize json because %s:\n%s", error, source);+            }++            // we only check the first one, since there are extra checks+            // in readNextTupleObject/readNextTupleList+            if (jsonSource[0].is_array()) {+                useObjects = false;+            } else if (jsonSource[0].is_object()) {+                useObjects = true;+                size_t index_pos = 0;+                for (auto param : params["relation"]["params"].array_items()) {+                    paramIndex.insert(std::make_pair(param.string_value(), index_pos));+                    index_pos++;+                }+            } else {+                fatal("the input is neither list nor object format");+            }+        }++        if (useObjects) {+            return readNextTupleObject();+        } else {+            return readNextTupleList();+        }+    }++    std::unique_ptr<RamDomain[]> readNextTupleList() {+        if (pos >= jsonSource.array_items().size()) {+            return nullptr;+        }++        std::unique_ptr<RamDomain[]> tuple = std::make_unique<RamDomain[]>(typeAttributes.size());+        const Json& jsonObj = jsonSource[pos];+        assert(jsonObj.is_array() && "the input is not json array");+        pos++;+        for (size_t i = 0; i < typeAttributes.size(); ++i) {+            try {+                auto&& ty = typeAttributes.at(i);+                switch (ty[0]) {+                    case 's': {+                        tuple[i] = symbolTable.unsafeLookup(jsonObj[i].string_value());+                        break;+                    }+                    case 'r': {+                        tuple[i] = readNextElementList(jsonObj[i], ty);+                        break;+                    }+                    case 'i': {+                        tuple[i] = jsonObj[i].int_value();+                        break;+                    }+                    case 'u': {+                        tuple[i] = jsonObj[i].int_value();+                        break;+                    }+                    case 'f': {+                        tuple[i] = jsonObj[i].number_value();+                        break;+                    }+                    default: fatal("invalid type attribute: `%c`", ty[0]);+                }+            } catch (...) {+                std::stringstream errorMessage;+                if (jsonObj.is_array() && i < jsonObj.array_items().size()) {+                    errorMessage << "Error converting: " << jsonObj[i].dump();+                } else {+                    errorMessage << "Invalid index: " << i;+                }+                throw std::invalid_argument(errorMessage.str());+            }+        }++        return tuple;+    }++    RamDomain readNextElementList(const Json& source, const std::string& recordTypeName) {+        auto&& recordInfo = types["records"][recordTypeName];++        if (recordInfo.is_null()) {+            throw std::invalid_argument("Missing record type information: " + recordTypeName);+        }++        // Handle null case+        if (source.is_null()) {+            return 0;+        }++        assert(source.is_array() && "the input is not json array");+        auto&& recordTypes = recordInfo["types"];+        const size_t recordArity = recordInfo["arity"].long_value();+        std::vector<RamDomain> recordValues(recordArity);+        for (size_t i = 0; i < recordArity; ++i) {+            const std::string& recordType = recordTypes[i].string_value();+            switch (recordType[0]) {+                case 's': {+                    recordValues[i] = symbolTable.unsafeLookup(source[i].string_value());+                    break;+                }+                case 'r': {+                    recordValues[i] = readNextElementList(source[i], recordType);+                    break;+                }+                case 'i': {+                    recordValues[i] = source[i].int_value();+                    break;+                }+                case 'u': {+                    recordValues[i] = source[i].int_value();+                    break;+                }+                case 'f': {+                    recordValues[i] = source[i].number_value();+                    break;+                }+                default: fatal("invalid type attribute");+            }+        }++        return recordTable.pack(recordValues.data(), recordValues.size());+    }++    std::unique_ptr<RamDomain[]> readNextTupleObject() {+        if (pos >= jsonSource.array_items().size()) {+            return nullptr;+        }++        std::unique_ptr<RamDomain[]> tuple = std::make_unique<RamDomain[]>(typeAttributes.size());+        const Json& jsonObj = jsonSource[pos];+        assert(jsonObj.is_object() && "the input is not json object");+        pos++;+        for (auto p : jsonObj.object_items()) {+            try {+                // get the corresponding position by parameter name+                if (paramIndex.find(p.first) == paramIndex.end()) {+                    fatal("invalid parameter: %s", p.first);+                }+                size_t i = paramIndex.at(p.first);+                auto&& ty = typeAttributes.at(i);+                switch (ty[0]) {+                    case 's': {+                        tuple[i] = symbolTable.unsafeLookup(p.second.string_value());+                        break;+                    }+                    case 'r': {+                        tuple[i] = readNextElementObject(p.second, ty);+                        break;+                    }+                    case 'i': {+                        tuple[i] = p.second.int_value();+                        break;+                    }+                    case 'u': {+                        tuple[i] = p.second.int_value();+                        break;+                    }+                    case 'f': {+                        tuple[i] = p.second.number_value();+                        break;+                    }+                    default: fatal("invalid type attribute: `%c`", ty[0]);+                }+            } catch (...) {+                std::stringstream errorMessage;+                errorMessage << "Error converting: " << p.second.dump();+                throw std::invalid_argument(errorMessage.str());+            }+        }++        return tuple;+    }++    RamDomain readNextElementObject(const Json& source, const std::string& recordTypeName) {+        auto&& recordInfo = types["records"][recordTypeName];+        const std::string recordName = recordTypeName.substr(2);+        std::map<const std::string, const size_t> recordIndex;++        size_t index_pos = 0;+        for (auto param : params["records"][recordName]["params"].array_items()) {+            recordIndex.insert(std::make_pair(param.string_value(), index_pos));+            index_pos++;+        }++        if (recordInfo.is_null()) {+            throw std::invalid_argument("Missing record type information: " + recordTypeName);+        }++        // Handle null case+        if (source.is_null()) {+            return 0;+        }++        assert(source.is_object() && "the input is not json object");+        auto&& recordTypes = recordInfo["types"];+        const size_t recordArity = recordInfo["arity"].long_value();+        std::vector<RamDomain> recordValues(recordArity);+        recordValues.reserve(recordIndex.size());+        for (auto readParam : source.object_items()) {+            // get the corresponding position by parameter name+            if (recordIndex.find(readParam.first) == recordIndex.end()) {+                fatal("invalid parameter: %s", readParam.first);+            }+            size_t i = recordIndex.at(readParam.first);+            auto&& type = recordTypes[i].string_value();+            switch (type[0]) {+                case 's': {+                    recordValues[i] = symbolTable.unsafeLookup(readParam.second.string_value());+                    break;+                }+                case 'r': {+                    recordValues[i] = readNextElementObject(readParam.second, type);+                    break;+                }+                case 'i': {+                    recordValues[i] = readParam.second.int_value();+                    break;+                }+                case 'u': {+                    recordValues[i] = readParam.second.int_value();+                    break;+                }+                case 'f': {+                    recordValues[i] = readParam.second.number_value();+                    break;+                }+                default: fatal("invalid type attribute: `%c`", type[0]);+            }+        }++        return recordTable.pack(recordValues.data(), recordValues.size());+    }+};++class ReadFileJSON : public ReadStreamJSON {+public:+    ReadFileJSON(const std::map<std::string, std::string>& rwOperation, SymbolTable& symbolTable,+            RecordTable& recordTable)+            : ReadStreamJSON(fileHandle, rwOperation, symbolTable, recordTable),+              baseName(souffle::baseName(getFileName(rwOperation))),+              fileHandle(getFileName(rwOperation), std::ios::in | std::ios::binary) {+        if (!fileHandle.is_open()) {+            throw std::invalid_argument("Cannot open json file " + baseName + "\n");+        }+    }++    ~ReadFileJSON() override = default;++protected:+    /**+     * Return given filename or construct from relation name.+     * Default name is [configured path]/[relation name].json+     *+     * @param rwOperation map of IO configuration options+     * @return input filename+     */+    static std::string getFileName(const std::map<std::string, std::string>& rwOperation) {+        auto name = getOr(rwOperation, "filename", rwOperation.at("name") + ".json");+        if (name.front() != '/') {+            name = getOr(rwOperation, "fact-dir", ".") + "/" + name;+        }+        return name;+    }++    std::string baseName;+    std::ifstream fileHandle;+};++class ReadCinJSONFactory : public ReadStreamFactory {+public:+    std::unique_ptr<ReadStream> getReader(const std::map<std::string, std::string>& rwOperation,+            SymbolTable& symbolTable, RecordTable& recordTable) override {+        return std::make_unique<ReadStreamJSON>(std::cin, rwOperation, symbolTable, recordTable);+    }++    const std::string& getName() const override {+        static const std::string name = "json";+        return name;+    }+    ~ReadCinJSONFactory() override = default;+};++class ReadFileJSONFactory : public ReadStreamFactory {+public:+    std::unique_ptr<ReadStream> getReader(const std::map<std::string, std::string>& rwOperation,+            SymbolTable& symbolTable, RecordTable& recordTable) override {+        return std::make_unique<ReadFileJSON>(rwOperation, symbolTable, recordTable);+    }++    const std::string& getName() const override {+        static const std::string name = "jsonfile";+        return name;+    }++    ~ReadFileJSONFactory() override = default;+};+}  // namespace souffle

@@ -36,7 +36,7 @@ public:     ReadStreamSQLite(const std::map<std::string, std::string>& rwOperation, SymbolTable& symbolTable,             RecordTable& recordTable)-            : ReadStream(rwOperation, symbolTable, recordTable), dbFilename(rwOperation.at("filename")),+            : ReadStream(rwOperation, symbolTable, recordTable), dbFilename(getFileName(rwOperation)),               relationName(rwOperation.at("name")) {         openDB();         checkTableExists();@@ -152,8 +152,28 @@         throw std::invalid_argument(                 "Required table or view does not exist in " + dbFilename + " for relation " + relationName);     }-    const std::string& dbFilename;-    const std::string& relationName;++    /**+     * Return given filename or construct from relation name.+     * Default name is [configured path]/[relation name].sqlite+     *+     * @param rwOperation map of IO configuration options+     * @return input filename+     */+    static std::string getFileName(const std::map<std::string, std::string>& rwOperation) {+        // legacy support for SQLite prior to 2020-03-18+        // convert dbname to filename+        auto name = getOr(rwOperation, "dbname", rwOperation.at("name") + ".sqlite");+        name = getOr(rwOperation, "filename", name);++        if (name.front() != '/') {+            name = getOr(rwOperation, "fact-dir", ".") + "/" + name;+        }+        return name;+    }++    const std::string dbFilename;+    const std::string relationName;     sqlite3_stmt* selectStatement = nullptr;     sqlite3* db = nullptr; };

@@ -48,22 +48,18 @@               typeAttributes(std::move(relTypes)), arity(typeAttributes.size() - auxArity),               auxiliaryArity(auxArity) {} -    SerialisationStream(-            RO<SymbolTable>& symTab, RO<RecordTable>& recTab, Json types, char const* const relationName)+    SerialisationStream(RO<SymbolTable>& symTab, RO<RecordTable>& recTab, Json types)             : symbolTable(symTab), recordTable(recTab), types(std::move(types)) {-        setupFromJson(relationName);+        setupFromJson();     }      SerialisationStream(RO<SymbolTable>& symTab, RO<RecordTable>& recTab,             const std::map<std::string, std::string>& rwOperation)             : symbolTable(symTab), recordTable(recTab) {-        auto&& relationName = rwOperation.at("name");-         std::string parseErrors;         types = Json::parse(rwOperation.at("types"), parseErrors);         assert(parseErrors.size() == 0 && "Internal JSON parsing failed.");--        setupFromJson(relationName.c_str());+        setupFromJson();     }      RO<SymbolTable>& symbolTable;@@ -75,8 +71,8 @@     size_t auxiliaryArity = 0;  private:-    void setupFromJson(char const* const relationName) {-        auto&& relInfo = types[relationName];+    void setupFromJson() {+        auto&& relInfo = types["relation"];         arity = static_cast<size_t>(relInfo["arity"].long_value());         auxiliaryArity = static_cast<size_t>(relInfo["auxArity"].long_value()); 

@@ -309,9 +309,9 @@     /**      * Get the attribute type of a relation at the column specified by the parameter.      * The attribute type is in the form "<primitive type>:<type name>".-     * <primitive type> can be s or n standing for symbol or number which are two primitive types in Souffle.-     * <type name> is the name given by the user in the Souffle Program after ".type", "symbol_type" or-     * "number_type". For example, for ".type Node", the type name is "Node".+     * <primitive type> can be s, f, u, or i standing for symbol, float, unsigned, and integer respectively,+     * which are the primitive types in Souffle.+     * <type name> is the name given by the user in the Souffle program      *      * @param The index of the column starting starting from 0 (size_t)      * @return The constant string of the attribute type@@ -680,6 +680,9 @@      * allRelations store all the relation in a vector.      */     std::vector<Relation*> allRelations;+    /**+     * The number of threads used by OpenMP+     */     std::size_t numThreads = 1;  protected:@@ -723,30 +726,28 @@      /**      * Execute program, loading inputs and storing outputs as required.-     * Read all input relations and store all output relations from the given directory.-     * First argument is the input directory, second argument is the output directory,-     * if no directory is given, the default is to use the current working directory.-     * The implementation of this function occurs in the C++ code generated by Souffle.-     * To view the generated C++ code, run Souffle with the `-g` option.+     * File IO types can use the given directories to find their input file.+     *+     * @param inputDirectory If non-empty, specifies the input directory+     * @param outputDirectory If non-empty, specifies the output directory      */-    virtual void runAll(std::string inputDirectory = ".", std::string outputDirectory = ".") = 0;+    virtual void runAll(std::string inputDirectory = "", std::string outputDirectory = "") = 0;      /**-     * Read all input relations from the given directory. If no directory is given, the-     * default is to use the current working directory. The implementation of this function-     * occurs in the C++ code generated by Souffle. To view the generated C++ code, run-     * Souffle with the `-g` option.+     * Read all input relations.+     *+     * File IO types can use the given directory to find their input file.+     * @param inputDirectory If non-empty, specifies the input directory      */-    virtual void loadAll(std::string inputDirectory = ".") = 0;+    virtual void loadAll(std::string inputDirectory = "") = 0;      /**-     * Store all output relations individually in .CSV file in the given directory, with-     * the relation name as the file name of the .CSV file. If no directory is given, the-     * default is to use the current working directory. The implementation of this function-     * occurs in the C++ code generated by Souffle. To view the generated C++ code, run-     * Souffle with the `-g` option.+     * Store all output relations.+     *+     * File IO types can use the given directory to find their input file.+     * @param outputDirectory If non-empty, specifies the output directory      */-    virtual void printAll(std::string outputDirectory = ".") = 0;+    virtual void printAll(std::string outputDirectory = "") = 0;      /**      * Output all the input relations in stdout, without generating any files. (for debug purposes).

@@ -25,6 +25,7 @@ #endif  #include <cstddef>+#include <iomanip> #include <iostream> #include <map> #include <ostream>@@ -72,10 +73,11 @@     WriteFileCSV(const std::map<std::string, std::string>& rwOperation, const SymbolTable& symbolTable,             const RecordTable& recordTable)             : WriteStreamCSV(rwOperation, symbolTable, recordTable),-              file(rwOperation.at("filename"), std::ios::out | std::ios::binary) {+              file(getFileName(rwOperation), std::ios::out | std::ios::binary) {         if (getOr(rwOperation, "headers", "false") == "true") {             file << rwOperation.at("attributeNames") << std::endl;         }+        file << std::setprecision(std::numeric_limits<RamFloat>::max_digits10);     }      ~WriteFileCSV() override = default;@@ -90,6 +92,21 @@     void writeNextTuple(const RamDomain* tuple) override {         writeNextTupleCSV(file, tuple);     }++    /**+     * Return given filename or construct from relation name.+     * Default name is [configured path]/[relation name].csv+     *+     * @param rwOperation map of IO configuration options+     * @return input filename+     */+    static std::string getFileName(const std::map<std::string, std::string>& rwOperation) {+        auto name = getOr(rwOperation, "filename", rwOperation.at("name") + ".csv");+        if (name.front() != '/') {+            name = getOr(rwOperation, "output-dir", ".") + "/" + name;+        }+        return name;+    } };  #ifdef USE_LIBZ@@ -98,10 +115,11 @@     WriteGZipFileCSV(const std::map<std::string, std::string>& rwOperation, const SymbolTable& symbolTable,             const RecordTable& recordTable)             : WriteStreamCSV(rwOperation, symbolTable, recordTable),-              file(rwOperation.at("filename"), std::ios::out | std::ios::binary) {+              file(getFileName(rwOperation), std::ios::out | std::ios::binary) {         if (getOr(rwOperation, "headers", "false") == "true") {             file << rwOperation.at("attributeNames") << std::endl;         }+        file << std::setprecision(std::numeric_limits<RamFloat>::max_digits10);     }      ~WriteGZipFileCSV() override = default;@@ -115,6 +133,21 @@         writeNextTupleCSV(file, tuple);     } +    /**+     * Return given filename or construct from relation name.+     * Default name is [configured path]/[relation name].csv+     *+     * @param rwOperation map of IO configuration options+     * @return input filename+     */+    static std::string getFileName(const std::map<std::string, std::string>& rwOperation) {+        auto name = getOr(rwOperation, "filename", rwOperation.at("name") + ".csv.gz");+        if (name.front() != '/') {+            name = getOr(rwOperation, "output-dir", ".") + "/" + name;+        }+        return name;+    }+     gzfstream::ogzfstream file; }; #endif@@ -129,6 +162,7 @@             std::cout << "\n" << rwOperation.at("attributeNames");         }         std::cout << "\n===============\n";+        std::cout << std::setprecision(std::numeric_limits<RamFloat>::max_digits10);     }      ~WriteCoutCSV() override {

@@ -0,0 +1,298 @@+/*+ * Souffle - A Datalog Compiler+ * Copyright (c) 2020, The Souffle Developers. All rights reserved+ * Licensed under the Universal Permissive License v 1.0 as shown at:+ * - https://opensource.org/licenses/UPL+ * - <souffle root>/licenses/SOUFFLE-UPL.txt+ */++/************************************************************************+ *+ * @file WriteStreamJSON.h+ *+ ***********************************************************************/++#pragma once++#include "RamTypes.h"+#include "SymbolTable.h"+#include "WriteStream.h"+#include "json11.h"+#include "utility/ContainerUtil.h"++#include <map>+#include <ostream>+#include <queue>+#include <stack>+#include <string>+#include <variant>+#include <vector>++namespace souffle {++class WriteStreamJSON : public WriteStream {+protected:+    WriteStreamJSON(const std::map<std::string, std::string>& rwOperation, const SymbolTable& symbolTable,+            const RecordTable& recordTable)+            : WriteStream(rwOperation, symbolTable, recordTable),+              useObjects(getOr(rwOperation, "format", "list") == "object") {+        if (useObjects) {+            std::string err;+            params = Json::parse(rwOperation.at("params"), err);+            if (err.length() > 0) {+                fatal("cannot get internal param names: %s", err);+            }+        }+    };++    const bool useObjects;+    Json params;++    void writeNextTupleJSON(std::ostream& destination, const RamDomain* tuple) {+        std::vector<Json> result;++        if (useObjects)+            destination << "{";+        else+            destination << "[";++        for (size_t col = 0; col < arity; ++col) {+            if (col > 0) {+                destination << ", ";+            }++            if (useObjects) {+                destination << params["relation"]["params"][col].dump() << ": ";+                writeNextTupleObject(destination, typeAttributes.at(col), tuple[col]);+            } else {+                writeNextTupleList(destination, typeAttributes.at(col), tuple[col]);+            }+        }++        if (useObjects)+            destination << "}";+        else+            destination << "]";+    }++    void writeNextTupleList(std::ostream& destination, const std::string& name, const RamDomain value) {+        using ValueTuple = std::pair<const std::string, const RamDomain>;+        std::stack<std::variant<ValueTuple, std::string>> worklist;+        worklist.push(std::make_pair(name, value));++        // the Json11 output is not tail recursive, therefore highly inefficient for recursive record+        // in addition the JSON object is immutable, so has memory overhead+        while (!worklist.empty()) {+            std::variant<ValueTuple, std::string> curr = worklist.top();+            worklist.pop();++            if (std::holds_alternative<std::string>(curr)) {+                destination << std::get<std::string>(curr);+                continue;+            }++            const std::string& currType = std::get<ValueTuple>(curr).first;+            const RamDomain currValue = std::get<ValueTuple>(curr).second;+            assert(currType.length() > 2 && "Invalid type length");+            switch (currType[0]) {+                // since some strings may need to be escaped, we use dump here+                case 's': destination << Json(symbolTable.unsafeResolve(currValue)).dump(); break;+                case 'i': destination << currValue; break;+                case 'u': destination << (int)ramBitCast<RamUnsigned>(currValue); break;+                case 'f': destination << ramBitCast<RamFloat>(currValue); break;+                case 'r': {+                    auto&& recordInfo = types["records"][currType];+                    assert(!recordInfo.is_null() && "Missing record type information");+                    if (currValue == 0) {+                        destination << "null";+                        break;+                    }++                    auto&& recordTypes = recordInfo["types"];+                    const size_t recordArity = recordInfo["arity"].long_value();+                    const RamDomain* tuplePtr = recordTable.unpack(currValue, recordArity);+                    worklist.push("]");+                    for (auto i = (long long)(recordArity - 1); i >= 0; --i) {+                        if (i != (long long)(recordArity - 1)) {+                            worklist.push(", ");+                        }+                        const std::string& recordType = recordTypes[i].string_value();+                        const RamDomain recordValue = tuplePtr[i];+                        worklist.push(std::make_pair(recordType, recordValue));+                    }++                    worklist.push("[");+                    break;+                }+                default: fatal("unsupported type attribute: `%c`", currType[0]);+            }+        }+    }++    void writeNextTupleObject(std::ostream& destination, const std::string& name, const RamDomain value) {+        using ValueTuple = std::pair<const std::string, const RamDomain>;+        std::stack<std::variant<ValueTuple, std::string>> worklist;+        worklist.push(std::make_pair(name, value));++        // the Json11 output is not tail recursive, therefore highly inefficient for recursive record+        // in addition the JSON object is immutable, so has memory overhead+        while (!worklist.empty()) {+            std::variant<ValueTuple, std::string> curr = worklist.top();+            worklist.pop();++            if (std::holds_alternative<std::string>(curr)) {+                destination << std::get<std::string>(curr);+                continue;+            }++            const std::string& currType = std::get<ValueTuple>(curr).first;+            const RamDomain currValue = std::get<ValueTuple>(curr).second;+            const std::string& typeName = currType.substr(2);+            assert(currType.length() > 2 && "Invalid type length");+            switch (currType[0]) {+                // since some strings may need to be escaped, we use dump here+                case 's': destination << Json(symbolTable.unsafeResolve(currValue)).dump(); break;+                case 'i': destination << currValue; break;+                case 'u': destination << (int)ramBitCast<RamUnsigned>(currValue); break;+                case 'f': destination << ramBitCast<RamFloat>(currValue); break;+                case 'r': {+                    auto&& recordInfo = types["records"][currType];+                    assert(!recordInfo.is_null() && "Missing record type information");+                    if (currValue == 0) {+                        destination << "null";+                        break;+                    }++                    auto&& recordTypes = recordInfo["types"];+                    const size_t recordArity = recordInfo["arity"].long_value();+                    const RamDomain* tuplePtr = recordTable.unpack(currValue, recordArity);+                    worklist.push("}");+                    for (auto i = (long long)(recordArity - 1); i >= 0; --i) {+                        if (i != (long long)(recordArity - 1)) {+                            worklist.push(", ");+                        }+                        const std::string& recordType = recordTypes[i].string_value();+                        const RamDomain recordValue = tuplePtr[i];+                        worklist.push(std::make_pair(recordType, recordValue));+                        worklist.push(": ");++                        auto&& recordParam = params["records"][typeName]["params"][i];+                        assert(recordParam.is_string());+                        worklist.push(recordParam.dump());+                    }++                    worklist.push("{");+                    break;+                }+                default: fatal("unsupported type attribute: `%c`", currType[0]);+            }+        }+    }+};++class WriteFileJSON : public WriteStreamJSON {+public:+    WriteFileJSON(const std::map<std::string, std::string>& rwOperation, const SymbolTable& symbolTable,+            const RecordTable& recordTable)+            : WriteStreamJSON(rwOperation, symbolTable, recordTable), isFirst(true),+              file(getFileName(rwOperation), std::ios::out | std::ios::binary) {+        file << "[";+    }++    ~WriteFileJSON() override {+        file << "]\n";+        file.close();+    }++protected:+    bool isFirst;+    std::ofstream file;++    void writeNullary() override {+        file << "null\n";+    }++    void writeNextTuple(const RamDomain* tuple) override {+        if (!isFirst) {+            file << ",\n";+        } else {+            isFirst = false;+        }+        writeNextTupleJSON(file, tuple);+    }++    /**+     * Return given filename or construct from relation name.+     * Default name is [configured path]/[relation name].json+     *+     * @param rwOperation map of IO configuration options+     * @return input filename+     */+    static std::string getFileName(const std::map<std::string, std::string>& rwOperation) {+        auto name = getOr(rwOperation, "filename", rwOperation.at("name") + ".json");+        if (name.front() != '/') {+            name = getOr(rwOperation, "output-dir", ".") + "/" + name;+        }+        return name;+    }+};++class WriteCoutJSON : public WriteStreamJSON {+public:+    WriteCoutJSON(const std::map<std::string, std::string>& rwOperation, const SymbolTable& symbolTable,+            const RecordTable& recordTable)+            : WriteStreamJSON(rwOperation, symbolTable, recordTable), isFirst(true) {+        std::cout << "[";+    }++    ~WriteCoutJSON() override {+        std::cout << "]\n";+    };++protected:+    bool isFirst;++    void writeNullary() override {+        std::cout << "null\n";+    }++    void writeNextTuple(const RamDomain* tuple) override {+        if (!isFirst) {+            std::cout << ",\n";+        } else {+            isFirst = false;+        }+        writeNextTupleJSON(std::cout, tuple);+    }+};++class WriteFileJSONFactory : public WriteStreamFactory {+public:+    std::unique_ptr<WriteStream> getWriter(const std::map<std::string, std::string>& rwOperation,+            const SymbolTable& symbolTable, const RecordTable& recordTable) override {+        return std::make_unique<WriteFileJSON>(rwOperation, symbolTable, recordTable);+    }++    const std::string& getName() const override {+        static const std::string name = "jsonfile";+        return name;+    }++    ~WriteFileJSONFactory() override = default;+};++class WriteCoutJSONFactory : public WriteStreamFactory {+public:+    std::unique_ptr<WriteStream> getWriter(const std::map<std::string, std::string>& rwOperation,+            const SymbolTable& symbolTable, const RecordTable& recordTable) override {+        return std::make_unique<WriteCoutJSON>(rwOperation, symbolTable, recordTable);+    }++    const std::string& getName() const override {+        static const std::string name = "json";+        return name;+    }++    ~WriteCoutJSONFactory() override = default;+};+}  // namespace souffle

@@ -37,7 +37,7 @@ public:     WriteStreamSQLite(const std::map<std::string, std::string>& rwOperation, const SymbolTable& symbolTable,             const RecordTable& recordTable)-            : WriteStream(rwOperation, symbolTable, recordTable), dbFilename(rwOperation.at("filename")),+            : WriteStream(rwOperation, symbolTable, recordTable), dbFilename(getFileName(rwOperation)),               relationName(rwOperation.at("name")) {         openDB();         createTables();@@ -250,8 +250,27 @@         executeSQL(createTableText.str(), db);     } -    const std::string& dbFilename;-    const std::string& relationName;+    /**+     * Return given filename or construct from relation name.+     * Default name is [configured path]/[relation name].sqlite+     *+     * @param rwOperation map of IO configuration options+     * @return input filename+     */+    static std::string getFileName(const std::map<std::string, std::string>& rwOperation) {+        // legacy support for SQLite prior to 2020-03-18+        // convert dbname to filename+        auto name = getOr(rwOperation, "dbname", rwOperation.at("name") + ".sqlite");+        name = getOr(rwOperation, "filename", name);++        if (name.front() != '/') {+            name = getOr(rwOperation, "output-dir", ".") + "/" + name;+        }+        return name;+    }++    const std::string dbFilename;+    const std::string relationName;     const std::string symbolTableName = "__SymbolTable";      std::unordered_map<uint64_t, uint64_t> dbSymbolTable;

@@ -173,6 +173,119 @@ }  // -------------------------------------------------------------+//                            Iterators+// -------------------------------------------------------------++/**+ * A wrapper for an iterator obtaining pointers of a certain type,+ * dereferencing values before forwarding them to the consumer.+ *+ * @tparam Iter ... the type of wrapped iterator+ * @tparam T    ... the value to be accessed by the resulting iterator+ */+template <typename Iter, typename T = typename std::remove_pointer<typename Iter::value_type>::type>+struct IterDerefWrapper : public std::iterator<std::forward_iterator_tag, T> {+    /* The nested iterator. */+    Iter iter;++public:+    // some constructors+    IterDerefWrapper() = default;+    IterDerefWrapper(const Iter& iter) : iter(iter) {}++    // defaulted copy and move constructors+    IterDerefWrapper(const IterDerefWrapper&) = default;+    IterDerefWrapper(IterDerefWrapper&&) = default;++    // default assignment operators+    IterDerefWrapper& operator=(const IterDerefWrapper&) = default;+    IterDerefWrapper& operator=(IterDerefWrapper&&) = default;++    /* The equality operator as required by the iterator concept. */+    bool operator==(const IterDerefWrapper& other) const {+        return iter == other.iter;+    }++    /* The not-equality operator as required by the iterator concept. */+    bool operator!=(const IterDerefWrapper& other) const {+        return iter != other.iter;+    }++    /* The deref operator as required by the iterator concept. */+    const T& operator*() const {+        return **iter;+    }++    /* Support for the pointer operator. */+    const T* operator->() const {+        return &(**iter);+    }++    /* The increment operator as required by the iterator concept. */+    IterDerefWrapper& operator++() {+        ++iter;+        return *this;+    }+};++/**+ * A factory function enabling the construction of a dereferencing+ * iterator utilizing the automated deduction of template parameters.+ */+template <typename Iter>+IterDerefWrapper<Iter> derefIter(const Iter& iter) {+    return IterDerefWrapper<Iter>(iter);+}++/**+ * An iterator to be utilized if there is only a single element to iterate over.+ */+template <typename T>+class SingleValueIterator : public std::iterator<std::forward_iterator_tag, T> {+    T value;++    bool end = true;++public:+    SingleValueIterator() = default;++    SingleValueIterator(const T& value) : value(value), end(false) {}++    // a copy constructor+    SingleValueIterator(const SingleValueIterator& other) = default;++    // an assignment operator+    SingleValueIterator& operator=(const SingleValueIterator& other) = default;++    // the equality operator as required by the iterator concept+    bool operator==(const SingleValueIterator& other) const {+        // only equivalent if pointing to the end+        return end && other.end;+    }++    // the not-equality operator as required by the iterator concept+    bool operator!=(const SingleValueIterator& other) const {+        return !(*this == other);+    }++    // the deref operator as required by the iterator concept+    const T& operator*() const {+        return value;+    }++    // support for the pointer operator+    const T* operator->() const {+        return &value;+    }++    // the increment operator as required by the iterator concept+    SingleValueIterator& operator++() {+        end = true;+        return *this;+    }+};++// ------------------------------------------------------------- //                             Ranges // ------------------------------------------------------------- 

@@ -45,26 +45,16 @@     return runRange(from, to, A(from <= to ? 1 : -1), std::forward<F>(go)); } -namespace details { template <typename A>-A symbol2numeric(const std::string& s);--#define SYM_2_NUMERIC_OVERLOAD(ty)                \-    template <>                                   \-    ty symbol2numeric<ty>(const std::string& s) { \-        return ty##FromString(s);                 \-    }--SYM_2_NUMERIC_OVERLOAD(RamFloat)-SYM_2_NUMERIC_OVERLOAD(RamSigned)-SYM_2_NUMERIC_OVERLOAD(RamUnsigned)-#undef SYM_2_NUMERIC_OVERLOAD-}  // namespace details--template <typename A> A symbol2numeric(const std::string& src) {     try {-        return details::symbol2numeric<A>(src);+        if constexpr (std::is_same_v<RamFloat, A>) {+            return RamFloatFromString(src);+        } else if constexpr (std::is_same_v<RamSigned, A>) {+            return RamSignedFromString(src);+        } else if constexpr (std::is_same_v<RamUnsigned, A>) {+            return RamUnsignedFromString(src);+        }     } catch (...) {         tfm::format(std::cerr, "error: wrong string provided by `to_number(\"%s\")` functor.\n", src);         raise(SIGFPE);

@@ -91,20 +91,24 @@  * Simple implementation of a which tool  */ inline std::string which(const std::string& name) {-    char buf[PATH_MAX];-    if ((::realpath(name.c_str(), buf) != nullptr) && isExecutable(buf)) {-        return buf;+    // Check if name has path components in it and if so return it immediately+    if (name.find('/') != std::string::npos) {+        return name;     }+    // Get PATH from environment, if it exists.     const char* syspath = ::getenv("PATH");     if (syspath == nullptr) {         return "";     }+    char buf[PATH_MAX];     std::stringstream sstr;     sstr << syspath;     std::string sub;++    // Check for existence of a binary called 'name' in PATH     while (std::getline(sstr, sub, ':')) {         std::string path = sub + "/" + name;-        if (isExecutable(path) && (realpath(path.c_str(), buf) != nullptr)) {+        if ((::realpath(path.c_str(), buf) != nullptr) && isExecutable(path) && !existDir(path)) {             return buf;         }     }

@@ -125,5 +125,4 @@  // HACK:  Workaround to suppress spurious reachability warnings. #define UNREACHABLE_BAD_CASE_ANALYSIS fatal("unhandled switch branch");- }  // namespace souffle

@@ -1,7 +0,0 @@--- | This module functions as a re-export for the compiled, more performant---   variant of the API available in this library (due to legacy reasons).-module Language.Souffle-  ( module Language.Souffle.Compiled-  ) where--import Language.Souffle.Compiled

@@ -1,5 +1,5 @@ {-# OPTIONS_GHC -Wno-redundant-constraints #-}-{-# LANGUAGE TypeFamilies, DerivingVia, InstanceSigs, BangPatterns #-}+{-# LANGUAGE FlexibleInstances, TypeFamilies, DerivingVia, InstanceSigs, BangPatterns #-}  -- | This module provides an implementation for the typeclasses defined in --   "Language.Souffle.Class".@@ -27,6 +27,9 @@ import Control.Monad.Reader import Data.Foldable ( traverse_ ) import Data.Proxy+import qualified Data.Array as A+import qualified Data.Array.IO as A+import qualified Data.Array.Unsafe as A import qualified Data.Vector as V import qualified Data.Vector.Mutable as MV import Foreign.ForeignPtr@@ -60,22 +63,42 @@ {-# INLINABLE runM #-}  instance MonadPush CMarshal where-  pushInt int = do+  pushInt32 int = do     tuple <- ask-    liftIO $ Internal.tuplePushInt tuple int-  {-# INLINABLE pushInt #-}+    liftIO $ Internal.tuplePushInt32 tuple int+  {-# INLINABLE pushInt32 #-} +  pushUInt32 int = do+    tuple <- ask+    liftIO $ Internal.tuplePushUInt32 tuple int+  {-# INLINABLE pushUInt32 #-}++  pushFloat float = do+    tuple <- ask+    liftIO $ Internal.tuplePushFloat tuple float+  {-# INLINABLE pushFloat #-}+   pushString str = do     tuple <- ask     liftIO $ Internal.tuplePushString tuple str   {-# INLINABLE pushString #-}  instance MonadPop CMarshal where-  popInt = do+  popInt32 = do     tuple <- ask-    liftIO $ Internal.tuplePopInt tuple-  {-# INLINABLE popInt #-}+    liftIO $ Internal.tuplePopInt32 tuple+  {-# INLINABLE popInt32 #-} +  popUInt32 = do+    tuple <- ask+    liftIO $ Internal.tuplePopUInt32 tuple+  {-# INLINABLE popUInt32 #-}++  popFloat = do+    tuple <- ask+    liftIO $ Internal.tuplePopFloat tuple+  {-# INLINABLE popFloat #-}+   popString = do     tuple <- ask     liftIO $ Internal.tuplePopString tuple@@ -105,6 +128,25 @@         result <- runM pop tuple         MV.unsafeWrite vec idx result         go vec (idx + 1) count it+  {-# INLINABLE collect #-}++instance Collect (A.Array Int) where+  collect factCount iterator = do+    array <- A.newArray_ (0, factCount - 1)+    go array 0 factCount iterator+    where+      go :: Marshal a+         => A.IOArray Int a+         -> Int+         -> Int+         -> ForeignPtr Internal.RelationIterator+         -> IO (A.Array Int a)+      go array idx count _ | idx == count = A.unsafeFreeze array+      go array idx count it = do+        tuple <- Internal.relationIteratorNext it+        result <- runM pop tuple+        A.writeArray array idx result+        go array (idx + 1) count it   {-# INLINABLE collect #-}  instance MonadSouffle SouffleM where

@@ -25,9 +25,13 @@   , allocTuple   , addTuple   , containsTuple-  , tuplePushInt+  , tuplePushInt32+  , tuplePushUInt32+  , tuplePushFloat   , tuplePushString-  , tuplePopInt+  , tuplePopInt32+  , tuplePopUInt32+  , tuplePopFloat   , tuplePopString   ) where @@ -148,23 +152,49 @@ {-# INLINABLE containsTuple #-}  -- | Pushes an integer value into a tuple.-tuplePushInt :: Ptr Tuple -> Int32 -> IO ()-tuplePushInt tuple i = Bindings.tuplePushInt tuple (CInt i)-{-# INLINABLE tuplePushInt #-}+tuplePushInt32 :: Ptr Tuple -> Int32 -> IO ()+tuplePushInt32 tuple i = Bindings.tuplePushInt32 tuple (CInt i)+{-# INLINABLE tuplePushInt32 #-} +-- | Pushes an unsigned integer value into a tuple.+tuplePushUInt32 :: Ptr Tuple -> Word32 -> IO ()+tuplePushUInt32 tuple i = Bindings.tuplePushUInt32 tuple (CUInt i)+{-# INLINABLE tuplePushUInt32 #-}++-- | Pushes a float value into a tuple.+tuplePushFloat :: Ptr Tuple -> Float -> IO ()+tuplePushFloat tuple f = Bindings.tuplePushFloat tuple (CFloat f)+{-# INLINABLE tuplePushFloat #-}+ -- | Pushes a string value into a tuple. tuplePushString :: Ptr Tuple -> String -> IO () tuplePushString tuple str =   withCString str $ Bindings.tuplePushString tuple {-# INLINABLE tuplePushString #-} --- | Extracts an integer value from a tuple.-tuplePopInt :: Ptr Tuple -> IO Int32-tuplePopInt tuple = alloca $ \ptr -> do-  Bindings.tuplePopInt tuple ptr+-- | Extracts a 32 bit signed integer value from a tuple.+tuplePopInt32 :: Ptr Tuple -> IO Int32+tuplePopInt32 tuple = alloca $ \ptr -> do+  Bindings.tuplePopInt32 tuple ptr   (CInt res) <- peek ptr   pure res-{-# INLINABLE tuplePopInt #-}+{-# INLINABLE tuplePopInt32 #-}++-- | Extracts a 32 bit unsigned integer value from a tuple.+tuplePopUInt32 :: Ptr Tuple -> IO Word32+tuplePopUInt32 tuple = alloca $ \ptr -> do+  Bindings.tuplePopUInt32 tuple ptr+  (CUInt res) <- peek ptr+  pure res+{-# INLINABLE tuplePopUInt32 #-}++-- | Extracts a float value from a tuple.+tuplePopFloat :: Ptr Tuple -> IO Float+tuplePopFloat tuple = alloca $ \ptr -> do+  Bindings.tuplePopFloat tuple ptr+  (CFloat res) <- peek ptr+  pure res+{-# INLINABLE tuplePopFloat #-}  -- | Extracts a string value from a tuple. tuplePopString :: Ptr Tuple -> IO String

@@ -23,10 +23,14 @@   , freeTuple   , addTuple   , containsTuple-  , tuplePushInt+  , tuplePushInt32+  , tuplePushUInt32   , tuplePushString-  , tuplePopInt+  , tuplePushFloat+  , tuplePopInt32+  , tuplePopUInt32   , tuplePopString+  , tuplePopFloat   ) where  import Prelude hiding ( init )@@ -204,7 +208,7 @@ foreign import ccall unsafe "souffle_contains_tuple" containsTuple   :: Ptr Relation -> Ptr Tuple -> IO CBool -{-| Pushes an integer value into a tuple.+{-| Pushes a 32 bit signed integer value into a tuple.      You need to check if the passed pointer is not equal to 'nullPtr' before     passing it to this function. Not doing so results in undefined behavior (in C++).@@ -213,9 +217,33 @@     in a crash. Pushing a value into a tuple when it already is "full"     also results in a crash. -}-foreign import ccall unsafe "souffle_tuple_push_int" tuplePushInt+foreign import ccall unsafe "souffle_tuple_push_int32" tuplePushInt32   :: Ptr Tuple -> CInt -> IO () +{-| Pushes a 32 bit unsigned integer value into a tuple.++    You need to check if the passed pointer is not equal to 'nullPtr' before+    passing it to this function. Not doing so results in undefined behavior (in C++).++    Pushing an integer value onto a tuple that expects another type results+    in a crash. Pushing a value into a tuple when it already is "full"+    also results in a crash.+-}+foreign import ccall unsafe "souffle_tuple_push_uint32" tuplePushUInt32+  :: Ptr Tuple -> CUInt -> IO ()++{-| Pushes a float value into a tuple.++    You need to check if the passed pointer is not equal to 'nullPtr' before+    passing it to this function. Not doing so results in undefined behavior (in C++).++    Pushing a float value onto a tuple that expects another type results+    in a crash. Pushing a value into a tuple when it already is "full"+    also results in a crash.+-}+foreign import ccall unsafe "souffle_tuple_push_float" tuplePushFloat+  :: Ptr Tuple -> CFloat -> IO ()+ {-| Pushes a string value into a tuple.      You need to check if the passed pointer is not equal to 'nullPtr' before@@ -228,7 +256,7 @@ foreign import ccall unsafe "souffle_tuple_push_string" tuplePushString   :: Ptr Tuple -> CString -> IO () -{-| Extracts an integer value from a tuple.+{-| Extracts a 32 bit signed integer value from a tuple.      You need to check if the passed pointer is not equal to 'nullPtr' before passing it     to this function. Not doing so results in undefined behavior.@@ -239,8 +267,36 @@      The popped integer will be stored in the pointer that is passed in. -}-foreign import ccall unsafe "souffle_tuple_pop_int" tuplePopInt+foreign import ccall unsafe "souffle_tuple_pop_int32" tuplePopInt32   :: Ptr Tuple -> Ptr CInt -> IO ()++{-| Extracts a 32 bit unsigned integer value from a tuple.++    You need to check if the passed pointer is not equal to 'nullPtr' before passing it+    to this function. Not doing so results in undefined behavior.++    Extracting an integer value from a tuple that expects another type results+    in a crash. Extracting a value from a tuple when it is already "empty"+    also results in a crash.++    The popped integer will be stored in the pointer that is passed in.+-}+foreign import ccall unsafe "souffle_tuple_pop_uint32" tuplePopUInt32+  :: Ptr Tuple -> Ptr CUInt -> IO ()++{-| Extracts a float value from a tuple.++    You need to check if the passed pointer is not equal to 'nullPtr' before passing it+    to this function. Not doing so results in undefined behavior.++    Extracting a float value from a tuple that expects another type results+    in a crash. Extracting a value from a tuple when it is already "empty"+    also results in a crash.++    The popped float will be stored in the pointer that is passed in.+-}+foreign import ccall unsafe "souffle_tuple_pop_float" tuplePopFloat+  :: Ptr Tuple -> Ptr CFloat -> IO ()  {-| Extracts a string value from a tuple. 

@@ -11,6 +11,7 @@ import GHC.Generics import Data.Kind import Data.Int+import Data.Word import qualified Data.Text as T import qualified Data.Text.Lazy as TL @@ -58,9 +59,11 @@   DirectlyMarshallable _ T.Text = ()   DirectlyMarshallable _ TL.Text = ()   DirectlyMarshallable _ Int32 = ()+  DirectlyMarshallable _ Word32 = ()+  DirectlyMarshallable _ Float = ()   DirectlyMarshallable _ String = ()   DirectlyMarshallable t a =     TypeError ( "Error while generating marshalling code for " <> t <> ":"-              % "Can only marshal values of Int32, String and Text directly"+              % "Can only marshal values of Int32, Word32, Float, String and Text directly"              <> ", but found " <> a <> " type instead.") 

@@ -1,5 +1,5 @@ {-# OPTIONS_GHC -Wno-redundant-constraints #-}-{-# LANGUAGE TypeFamilies, DerivingVia, InstanceSigs #-}+{-# LANGUAGE FlexibleInstances, TypeFamilies, DerivingVia, InstanceSigs #-}  -- | This module provides an implementation for the `MonadSouffle` typeclass --   defined in "Language.Souffle.Class".@@ -28,7 +28,7 @@ import Prelude hiding (init)  import Control.DeepSeq (deepseq)-import Control.Exception (ErrorCall(..), throwIO)+import Control.Exception (ErrorCall(..), throwIO, bracket) import Control.Monad.State.Strict import Control.Monad.Reader import Data.IORef@@ -37,6 +37,7 @@ import Data.Semigroup (Last(..)) import Data.Maybe (fromMaybe) import Data.Proxy+import qualified Data.Array as A import qualified Data.Text as T import qualified Data.Vector as V import Data.Word@@ -139,18 +140,34 @@   via (State [String])  instance MonadPush IMarshal where-  pushInt int = modify (show int:)-  {-# INLINABLE pushInt #-}+  pushInt32 int = modify (show int:)+  {-# INLINABLE pushInt32 #-} +  pushUInt32 int = modify (show int:)+  {-# INLINABLE pushUInt32 #-}++  pushFloat float = modify (show float:)+  {-# INLINABLE pushFloat #-}+   pushString str = modify (str:)   {-# INLINABLE pushString #-}  instance MonadPop IMarshal where-  popInt = state $ \case+  popInt32 = state $ \case     [] -> error "Empty fact stack"     (h:t) -> (read h, t)-  {-# INLINABLE popInt #-}+  {-# INLINABLE popInt32 #-} +  popUInt32 = state $ \case+    [] -> error "Empty fact stack"+    (h:t) -> (read h, t)+  {-# INLINABLE popUInt32 #-}++  popFloat = state $ \case+    [] -> error "Empty fact stack"+    (h:t) -> (read h, t)+  {-# INLINABLE popFloat #-}+   popString = state $ \case     [] -> error "Empty fact stack"     (h:t) -> (h, t)@@ -178,6 +195,13 @@   collect factFile = V.fromList <$!> collect factFile   {-# INLINABLE collect #-} +instance Collect (A.Array Int) where+  collect factFile = do+    facts <- collect factFile+    let count = length facts+    pure $! A.listArray (0, count - 1) facts+  {-# INLINABLE collect #-}+ instance MonadSouffle SouffleM where   type Handler SouffleM = Handle   type CollectFacts SouffleM c = Collect c@@ -226,18 +250,23 @@             , std_out = CreatePipe             , std_err = CreatePipe             }-    (_, mStdOutHandle, mStdErrHandle, processHandle) <- createProcess_ "souffle-haskell" processToRun-    waitForProcess processHandle >>= \case-      ExitSuccess   -> pure ()-      ExitFailure c -> throwIO $ ErrorCall $ "Souffle exited with: " ++ show c-    forM_ mStdOutHandle $ \stdoutHandle -> do-      stdout <- T.pack <$> hGetContents stdoutHandle-      writeIORef refHandleStdOut $! Just $! stdout-      hClose stdoutHandle-    forM_ mStdErrHandle $ \stderrHandle -> do-      stderr <- T.pack <$> hGetContents stderrHandle-      writeIORef refHandleStdErr $! Just $! stderr-      hClose stderrHandle+    bracket+      (createProcess_ "souffle-haskell" processToRun)+      (\(_, mStdOutHandle, mStdErrHandle, _) -> do+        traverse_ hClose mStdOutHandle+        traverse_ hClose mStdErrHandle+      )+      (\(_, mStdOutHandle, mStdErrHandle, processHandle) -> do+        waitForProcess processHandle >>= \case+          ExitSuccess   -> pure ()+          ExitFailure c -> throwIO $ ErrorCall $ "Souffle exited with: " ++ show c+        forM_ mStdOutHandle $ \stdoutHandle -> do+          stdout <- T.pack <$!> hGetContents stdoutHandle+          writeIORef refHandleStdOut $! Just $! stdout+        forM_ mStdErrHandle $ \stderrHandle -> do+          stderr <- T.pack <$!> hGetContents stderrHandle+          writeIORef refHandleStdErr $! Just $! stderr+      )   {-# INLINABLE run #-}    setNumThreads handle n = liftIO $

@@ -14,6 +14,7 @@  import GHC.Generics import Data.Int+import Data.Word import qualified Data.Text as T import qualified Data.Text.Lazy as TL import qualified Language.Souffle.Internal.Constraints as C@@ -25,8 +26,12 @@ See also: 'MonadPop', 'Marshal'. -} class Monad m => MonadPush m where-  -- | Marshals an integer to the datalog side.-  pushInt :: Int32 -> m ()+  -- | Marshals a signed 32 bit integer to the datalog side.+  pushInt32 :: Int32 -> m ()+  -- | Marshals an unsigned 32 bit integer to the datalog side.+  pushUInt32 :: Word32 -> m ()+  -- | Marshals a float to the datalog side.+  pushFloat :: Float -> m ()   -- | Marshals a string to the datalog side.   pushString :: String -> m () @@ -37,8 +42,12 @@ See also: 'MonadPush', 'Marshal'. -} class Monad m => MonadPop m where-  -- | Unmarshals an integer from the datalog side.-  popInt :: m Int32+  -- | Unmarshals a signed 32 bir integer from the datalog side.+  popInt32 :: m Int32+  -- | Unmarshals an unsigned 32 bir integer from the datalog side.+  popUInt32 :: m Word32+  -- | Unmarshals a float from the datalog side.+  popFloat :: m Float   -- | Unmarshals a string from the datalog side.   popString :: m String @@ -79,9 +88,21 @@   {-# INLINABLE pop #-}  instance Marshal Int32 where-  push = pushInt+  push = pushInt32   {-# INLINABLE push #-}-  pop = popInt+  pop = popInt32+  {-# INLINABLE pop #-}++instance Marshal Word32 where+  push = pushUInt32+  {-# INLINABLE push #-}+  pop = popUInt32+  {-# INLINABLE pop #-}++instance Marshal Float where+  push = pushFloat+  {-# INLINABLE push #-}+  pop = popFloat   {-# INLINABLE pop #-}  instance Marshal String where

@@ -4,10 +4,10 @@ -- -- see: https://github.com/sol/hpack ----- hash: e178bba53f0bfba290c3747f4551e6a6300a1f9a2fd84f691701b456b2001cc8+-- hash: 0e669f6fe266f6ab6d74d38303f2f74ccf8b6264c4aec9aeb972e5e76b38d236  name:           souffle-haskell-version:        1.0.0+version:        1.1.0 synopsis:       Souffle Datalog bindings for Haskell description:    Souffle Datalog bindings for Haskell. category:       Logic Programming, Foreign Binding, Bindings@@ -33,7 +33,6 @@     cbits/souffle/EventProcessor.h     cbits/souffle/gzfstream.h     cbits/souffle/IOSystem.h-    cbits/souffle/IterUtils.h     cbits/souffle/json11.h     cbits/souffle/LambdaBTree.h     cbits/souffle/Logger.h@@ -47,6 +46,7 @@     cbits/souffle/RamTypes.h     cbits/souffle/ReadStream.h     cbits/souffle/ReadStreamCSV.h+    cbits/souffle/ReadStreamJSON.h     cbits/souffle/ReadStreamSQLite.h     cbits/souffle/RecordTable.h     cbits/souffle/SerialisationStream.h@@ -67,6 +67,7 @@     cbits/souffle/utility/tinyformat.h     cbits/souffle/WriteStream.h     cbits/souffle/WriteStreamCSV.h+    cbits/souffle/WriteStreamJSON.h     cbits/souffle/WriteStreamSQLite.h     cbits/souffle/LICENSE @@ -76,7 +77,6 @@  library   exposed-modules:-      Language.Souffle       Language.Souffle.Class       Language.Souffle.Compiled       Language.Souffle.Internal@@ -121,9 +121,10 @@       souffle/LambdaBTree.h       souffle/BTree.h       souffle/utility/ParallelUtil.h+      souffle/WriteStreamJSON.h       souffle/WriteStreamCSV.h-      souffle/utility/tinyformat.h       souffle/IOSystem.h+      souffle/ReadStreamJSON.h       souffle/ReadStreamSQLite.h       souffle/UnionFind.h       souffle/PiggyList.h@@ -131,16 +132,17 @@       souffle/SerialisationStream.h       souffle/CompiledSouffle.h       souffle/EquivalenceRelation.h-      souffle/IterUtils.h       souffle/SignalHandler.h       souffle/Table.h       souffle/utility/EvaluatorUtil.h       souffle/CompiledOptions.h       souffle/Logger.h+      souffle/utility/tinyformat.h   cxx-sources:       cbits/souffle.cpp   build-depends:-      base >=4.12 && <5+      array <=1.0+    , base >=4.12 && <5     , deepseq >=1.4.4 && <2     , directory >=1.3.3 && <2     , filepath >=1.4.2 && <2@@ -193,9 +195,10 @@       souffle/LambdaBTree.h       souffle/BTree.h       souffle/utility/ParallelUtil.h+      souffle/WriteStreamJSON.h       souffle/WriteStreamCSV.h-      souffle/utility/tinyformat.h       souffle/IOSystem.h+      souffle/ReadStreamJSON.h       souffle/ReadStreamSQLite.h       souffle/UnionFind.h       souffle/PiggyList.h@@ -203,14 +206,15 @@       souffle/SerialisationStream.h       souffle/CompiledSouffle.h       souffle/EquivalenceRelation.h-      souffle/IterUtils.h       souffle/SignalHandler.h       souffle/Table.h       souffle/utility/EvaluatorUtil.h       souffle/CompiledOptions.h       souffle/Logger.h+      souffle/utility/tinyformat.h   build-depends:-      base >=4.12 && <5+      array <=1.0+    , base >=4.12 && <5     , containers >=0.6.2.1 && <1     , deepseq >=1.4.4 && <2     , directory >=1.3.3 && <2@@ -268,9 +272,10 @@       souffle/LambdaBTree.h       souffle/BTree.h       souffle/utility/ParallelUtil.h+      souffle/WriteStreamJSON.h       souffle/WriteStreamCSV.h-      souffle/utility/tinyformat.h       souffle/IOSystem.h+      souffle/ReadStreamJSON.h       souffle/ReadStreamSQLite.h       souffle/UnionFind.h       souffle/PiggyList.h@@ -278,20 +283,24 @@       souffle/SerialisationStream.h       souffle/CompiledSouffle.h       souffle/EquivalenceRelation.h-      souffle/IterUtils.h       souffle/SignalHandler.h       souffle/Table.h       souffle/utility/EvaluatorUtil.h       souffle/CompiledOptions.h       souffle/Logger.h+      souffle/utility/tinyformat.h   cxx-sources:       tests/fixtures/path.cpp+      tests/fixtures/round_trip.cpp   build-depends:-      base >=4.12 && <5+      array <=1.0+    , base >=4.12 && <5     , deepseq >=1.4.4 && <2     , directory >=1.3.3 && <2     , filepath >=1.4.2 && <2+    , hedgehog ==1.*     , hspec >=2.6.1 && <3.0.0+    , hspec-hedgehog ==0.*     , mtl >=2.0 && <3     , process >=1.6 && <2     , souffle-haskell

@@ -7,8 +7,9 @@ import Test.Hspec import GHC.Generics import Data.Maybe+import qualified Data.Array as A import qualified Data.Vector as V-import qualified Language.Souffle as Souffle+import qualified Language.Souffle.Compiled as Souffle  data Path = Path @@ -70,6 +71,16 @@         pure (es , rs)       edges `shouldBe` V.fromList [Edge "a" "b", Edge "b" "c"]       reachables `shouldBe` V.fromList [Reachable "a" "b", Reachable "a" "c", Reachable "b" "c"]++    it "can retrieve facts as an array" $ do+      (edges, reachables) <- Souffle.runSouffle $ do+        prog <- fromJust <$> Souffle.init Path+        Souffle.run prog+        es <- Souffle.getFacts prog+        rs <- Souffle.getFacts prog+        pure (es , rs)+      edges `shouldBe` A.listArray (0 :: Int, 1) [Edge "a" "b", Edge "b" "c"]+      reachables `shouldBe` A.listArray (0 :: Int, 2) [Reachable "a" "b", Reachable "a" "c", Reachable "b" "c"]      it "returns no facts if program hasn't run yet" $ do       edges <- Souffle.runSouffle $ do

@@ -12,6 +12,7 @@ import Control.Monad.IO.Class (liftIO) import System.Directory import System.IO.Temp+import qualified Data.Array as A import qualified Data.Vector as V import qualified Language.Souffle.Interpreted as Souffle @@ -90,6 +91,17 @@         pure (es , rs)       edges `shouldBe` V.fromList [Edge "a" "b", Edge "b" "c"]       reachables `shouldBe` V.fromList [Reachable "a" "b", Reachable "a" "c", Reachable "b" "c"]++    it "can retrieve facts as an array" $ do+      (edges, reachables) <- Souffle.runSouffle $ do+        prog <- fromJust <$> Souffle.init PathNoInput+        Souffle.run prog+        es <- Souffle.getFacts prog+        rs <- Souffle.getFacts prog+        Souffle.cleanup prog+        pure (es , rs)+      edges `shouldBe` A.listArray (0 :: Int, 1) [Edge "a" "b", Edge "b" "c"]+      reachables `shouldBe` A.listArray (0 :: Int, 2) [Reachable "a" "b", Reachable "a" "c", Reachable "b" "c"]      it "returns no facts if program hasn't run yet" $ do       edges <- Souffle.runSouffle $ do

@@ -1,20 +1,38 @@ -{-# LANGUAGE DeriveGeneric #-}+{-# LANGUAGE DeriveGeneric, TypeFamilies, DataKinds, RankNTypes #-}  module Test.Language.Souffle.MarshalSpec   ( module Test.Language.Souffle.MarshalSpec   ) where  import Test.Hspec+import Test.Hspec.Hedgehog+import qualified Hedgehog.Gen as Gen+import qualified Hedgehog.Range as Range import GHC.Generics-import Language.Souffle.Marshal+import qualified Data.Text as T+import qualified Data.Text.Lazy as TL import Data.Text import Data.Int+import Data.Word+import Data.Maybe ( fromJust )+import Control.Monad.IO.Class ( liftIO )+import Language.Souffle.Marshal+import qualified Language.Souffle.Marshal as Souffle+import qualified Language.Souffle.Class as Souffle+import qualified Language.Souffle.Compiled as Compiled+import qualified Language.Souffle.Interpreted as Interpreted   data Edge = Edge String String   deriving (Eq, Show, Generic) +newtype EdgeUInt = EdgeUInt Word32+  deriving (Eq, Show, Generic)++newtype FloatValue = FloatValue Float+  deriving (Eq, Show, Generic)+ data EdgeStrict = EdgeStrict !String !String   deriving (Eq, Show, Generic) @@ -49,6 +67,8 @@   instance Marshal Edge+instance Marshal EdgeUInt+instance Marshal FloatValue instance Marshal EdgeStrict instance Marshal EdgeUnpacked instance Marshal EdgeSynonyms@@ -59,8 +79,123 @@ instance Marshal LargeRecord  +data RoundTrip = RoundTrip++newtype StringFact = StringFact String+  deriving (Eq, Show, Generic)++newtype TextFact = TextFact T.Text+  deriving (Eq, Show, Generic)++newtype LazyTextFact = LazyTextFact TL.Text+  deriving (Eq, Show, Generic)++newtype Int32Fact = Int32Fact Int32+  deriving (Eq, Show, Generic)++newtype Word32Fact = Word32Fact Word32+  deriving (Eq, Show, Generic)++newtype FloatFact = FloatFact Float+  deriving (Eq, Show, Generic)++instance Souffle.Fact StringFact where+  factName = const "string_fact"++instance Souffle.Fact TextFact where+  factName = const "string_fact"++instance Souffle.Fact LazyTextFact where+  factName = const "string_fact"++instance Souffle.Fact Int32Fact where+  factName = const "number_fact"++instance Souffle.Fact Word32Fact where+  factName = const "unsigned_fact"++instance Souffle.Fact FloatFact where+  factName = const "float_fact"++instance Souffle.Marshal StringFact+instance Souffle.Marshal TextFact+instance Souffle.Marshal LazyTextFact+instance Souffle.Marshal Int32Fact+instance Souffle.Marshal Word32Fact+instance Souffle.Marshal FloatFact++instance Souffle.Program RoundTrip where+  type ProgramFacts RoundTrip = +    [StringFact, TextFact, LazyTextFact, Int32Fact, Word32Fact, FloatFact]+  programName = const "round_trip"++type RoundTripAction+  = forall a. (Souffle.Fact a, Souffle.ContainsFact RoundTrip a)+  => a -> PropertyT IO a+ spec :: Spec-spec = describe "Auto-deriving marshalling code" $-  it "can generate code for all instances in this file" $-    -- If this file compiles, then the test has already passed-    42 `shouldBe` 42+spec = describe "Marshalling" $ parallel $ do+  describe "Auto-deriving marshalling code" $+    it "can generate code for all instances in this file" $+      -- If this file compiles, then the test has already passed+      42 `shouldBe` 42++  describe "data transfer between Haskell and Souffle" $ parallel $ do+    let roundTripTests :: RoundTripAction -> Spec+        roundTripTests run = do+          it "can serialize and deserialize String values" $ hedgehog $ do+            str <- forAll $ Gen.string (Range.linear 0 10) Gen.unicode+            let fact = StringFact str+            fact' <- run fact+            fact === fact'++          it "can serialize and deserialize lazy Text" $ hedgehog $ do+            str <- forAll $ Gen.string (Range.linear 0 10) Gen.unicode+            let fact = LazyTextFact (TL.pack str)+            fact' <- run fact+            fact === fact'++          it "can serialize and deserialize strict Text values" $ hedgehog $ do+            str <- forAll $ Gen.text (Range.linear 0 10) Gen.unicode+            let fact = TextFact str+            fact' <- run fact+            fact === fact'++          it "can serialize and deserialize Int32 values" $ hedgehog $ do+            x <- forAll $ Gen.int32 (Range.linear minBound maxBound)+            let fact = Int32Fact x+            fact' <- run fact+            fact === fact'++          it "can serialize and deserialize Word32 values" $ hedgehog $ do+            x <- forAll $ Gen.word32 (Range.linear minBound maxBound)+            let fact = Word32Fact x+            fact' <- run fact+            fact === fact'++          {- TODO: enable this test once souffle floating point conversions are fixed+          it "can serialize and deserialize Float values" $ hedgehog $ do+            let epsilon = 1e-6+                fmin = -1e9+                fmax =  1e9+            x <- forAll $ Gen.float (Range.exponentialFloat fmin fmax)+            let fact = FloatFact x+            FloatFact x' <- run fact+            (abs (x' - x) < epsilon) === True+          -}++    describe "interpreted mode" $ parallel $+      roundTripTests $ \fact -> liftIO $ Interpreted.runSouffle $ do+        handle <- fromJust <$> Interpreted.init RoundTrip+        Interpreted.addFact handle fact+        Interpreted.run handle+        fact' <- Prelude.head <$> Interpreted.getFacts handle+        Interpreted.cleanup handle+        pure fact'++    describe "compiled mode" $ parallel $+      roundTripTests $ \fact -> liftIO $ Compiled.runSouffle $ do+        handle <- fromJust <$> Compiled.init RoundTrip+        Compiled.addFact handle fact+        Compiled.run handle+        Prelude.head <$> Compiled.getFacts handle

@@ -0,0 +1,399 @@++#include "souffle/CompiledSouffle.h"++extern "C" {+}++namespace souffle {+static const RamDomain RAM_BIT_SHIFT_MASK = RAM_DOMAIN_SIZE - 1;+struct t_btree_1__0__1 {+using t_tuple = Tuple<RamDomain, 1>;+struct t_comparator_0{+ int operator()(const t_tuple& a, const t_tuple& b) const {+  return (a[0] < b[0]) ? -1 : ((a[0] > b[0]) ? 1 :(0));+ }+bool less(const t_tuple& a, const t_tuple& b) const {+  return  a[0] < b[0];+ }+bool equal(const t_tuple& a, const t_tuple& b) const {+return a[0] == b[0];+ }+};+using t_ind_0 = btree_set<t_tuple,t_comparator_0>;+t_ind_0 ind_0;+using iterator = t_ind_0::iterator;+struct context {+t_ind_0::operation_hints hints_0;+};+context createContext() { return context(); }+bool insert(const t_tuple& t) {+context h;+return insert(t, h);+}+bool insert(const t_tuple& t, context& h) {+if (ind_0.insert(t, h.hints_0)) {+return true;+} else return false;+}+bool insert(const RamDomain* ramDomain) {+RamDomain data[1];+std::copy(ramDomain, ramDomain + 1, data);+const t_tuple& tuple = reinterpret_cast<const t_tuple&>(data);+context h;+return insert(tuple, h);+}+bool insert(RamDomain a0) {+RamDomain data[1] = {a0};+return insert(data);+}+bool contains(const t_tuple& t, context& h) const {+return ind_0.contains(t, h.hints_0);+}+bool contains(const t_tuple& t) const {+context h;+return contains(t, h);+}+std::size_t size() const {+return ind_0.size();+}+iterator find(const t_tuple& t, context& h) const {+return ind_0.find(t, h.hints_0);+}+iterator find(const t_tuple& t) const {+context h;+return find(t, h);+}+range<iterator> lowerUpperRange_0(const t_tuple& lower, const t_tuple& upper, context& h) const {+return range<iterator>(ind_0.begin(),ind_0.end());+}+range<iterator> lowerUpperRange_0(const t_tuple& lower, const t_tuple& upper) const {+return range<iterator>(ind_0.begin(),ind_0.end());+}+range<t_ind_0::iterator> lowerUpperRange_1(const t_tuple& lower, const t_tuple& upper, context& h) const {+auto pos = ind_0.find(lower, h.hints_0);+auto fin = ind_0.end();+if (pos != fin) {fin = pos; ++fin;}+return make_range(pos, fin);+}+range<t_ind_0::iterator> lowerUpperRange_1(const t_tuple& lower, const t_tuple& upper) const {+context h;+return lowerUpperRange_1(lower,upper,h);+}+bool empty() const {+return ind_0.empty();+}+std::vector<range<iterator>> partition() const {+return ind_0.getChunks(400);+}+void purge() {+ind_0.clear();+}+iterator begin() const {+return ind_0.begin();+}+iterator end() const {+return ind_0.end();+}+void printStatistics(std::ostream& o) const {+o << " arity 1 direct b-tree index 0 lex-order [0]\n";+ind_0.printStats(o);+}+};++class Sf_round_trip : public SouffleProgram {+private:+static inline bool regex_wrapper(const std::string& pattern, const std::string& text) {+   bool result = false; +   try { result = std::regex_match(text, std::regex(pattern)); } catch(...) { +     std::cerr << "warning: wrong pattern provided for match(\"" << pattern << "\",\"" << text << "\").\n";+}+   return result;+}+private:+static inline std::string substr_wrapper(const std::string& str, size_t idx, size_t len) {+   std::string result; +   try { result = str.substr(idx,len); } catch(...) { +     std::cerr << "warning: wrong index position provided by substr(\"";+     std::cerr << str << "\"," << (int32_t)idx << "," << (int32_t)len << ") functor.\n";+   } return result;+}+public:+// -- initialize symbol table --+SymbolTable symTable;// -- initialize record table --+RecordTable recordTable;+// -- Table: float_fact+std::unique_ptr<t_btree_1__0__1> rel_1_float_fact = std::make_unique<t_btree_1__0__1>();+souffle::RelationWrapper<0,t_btree_1__0__1,Tuple<RamDomain,1>,1,0> wrapper_rel_1_float_fact;+// -- Table: number_fact+std::unique_ptr<t_btree_1__0__1> rel_2_number_fact = std::make_unique<t_btree_1__0__1>();+souffle::RelationWrapper<1,t_btree_1__0__1,Tuple<RamDomain,1>,1,0> wrapper_rel_2_number_fact;+// -- Table: string_fact+std::unique_ptr<t_btree_1__0__1> rel_3_string_fact = std::make_unique<t_btree_1__0__1>();+souffle::RelationWrapper<2,t_btree_1__0__1,Tuple<RamDomain,1>,1,0> wrapper_rel_3_string_fact;+// -- Table: unsigned_fact+std::unique_ptr<t_btree_1__0__1> rel_4_unsigned_fact = std::make_unique<t_btree_1__0__1>();+souffle::RelationWrapper<3,t_btree_1__0__1,Tuple<RamDomain,1>,1,0> wrapper_rel_4_unsigned_fact;+public:+Sf_round_trip() : +wrapper_rel_1_float_fact(*rel_1_float_fact,symTable,"float_fact",std::array<const char *,1>{{"f:float"}},std::array<const char *,1>{{"x"}}),++wrapper_rel_2_number_fact(*rel_2_number_fact,symTable,"number_fact",std::array<const char *,1>{{"i:number"}},std::array<const char *,1>{{"x"}}),++wrapper_rel_3_string_fact(*rel_3_string_fact,symTable,"string_fact",std::array<const char *,1>{{"s:symbol"}},std::array<const char *,1>{{"x"}}),++wrapper_rel_4_unsigned_fact(*rel_4_unsigned_fact,symTable,"unsigned_fact",std::array<const char *,1>{{"u:unsigned"}},std::array<const char *,1>{{"x"}}){+addRelation("float_fact",&wrapper_rel_1_float_fact,true,true);+addRelation("number_fact",&wrapper_rel_2_number_fact,true,true);+addRelation("string_fact",&wrapper_rel_3_string_fact,true,true);+addRelation("unsigned_fact",&wrapper_rel_4_unsigned_fact,true,true);+}+~Sf_round_trip() {+}+private:+std::string inputDirectory;+std::string outputDirectory;+bool performIO;+std::atomic<RamDomain> ctr{};++std::atomic<size_t> iter{};+void runFunction(std::string inputDirectory = ".", std::string outputDirectory = ".", bool performIO = false) {+this->inputDirectory = inputDirectory;+this->outputDirectory = outputDirectory;+this->performIO = performIO;+SignalHandler::instance()->set();+#if defined(_OPENMP)+if (getNumThreads() > 0) {omp_set_num_threads(getNumThreads());}+#endif++// -- query evaluation --+{+ std::vector<RamDomain> args, ret;+subroutine_0(args, ret);+}+{+ std::vector<RamDomain> args, ret;+subroutine_1(args, ret);+}+{+ std::vector<RamDomain> args, ret;+subroutine_2(args, ret);+}+{+ std::vector<RamDomain> args, ret;+subroutine_3(args, ret);+}++// -- relation hint statistics --+SignalHandler::instance()->reset();+}+public:+void run() override { runFunction(".", ".", false); }+public:+void runAll(std::string inputDirectory = ".", std::string outputDirectory = ".") override { runFunction(inputDirectory, outputDirectory, true);+}+public:+void printAll(std::string outputDirectory = ".") override {+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"attributeNames","x"},{"filename","./string_fact.csv"},{"name","string_fact"},{"operation","output"},{"types","{\"records\": {}, \"string_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"s:symbol\"]}}"}});+if (!outputDirectory.empty() && directiveMap["IO"] == "file" && directiveMap["filename"].front() != '/') {directiveMap["filename"] = outputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getWriter(directiveMap, symTable, recordTable)->writeAll(*rel_3_string_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"attributeNames","x"},{"filename","./unsigned_fact.csv"},{"name","unsigned_fact"},{"operation","output"},{"types","{\"records\": {}, \"unsigned_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"u:unsigned\"]}}"}});+if (!outputDirectory.empty() && directiveMap["IO"] == "file" && directiveMap["filename"].front() != '/') {directiveMap["filename"] = outputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getWriter(directiveMap, symTable, recordTable)->writeAll(*rel_4_unsigned_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"attributeNames","x"},{"filename","./number_fact.csv"},{"name","number_fact"},{"operation","output"},{"types","{\"number_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"i:number\"]}, \"records\": {}}"}});+if (!outputDirectory.empty() && directiveMap["IO"] == "file" && directiveMap["filename"].front() != '/') {directiveMap["filename"] = outputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getWriter(directiveMap, symTable, recordTable)->writeAll(*rel_2_number_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"attributeNames","x"},{"filename","./float_fact.csv"},{"name","float_fact"},{"operation","output"},{"types","{\"float_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"f:float\"]}, \"records\": {}}"}});+if (!outputDirectory.empty() && directiveMap["IO"] == "file" && directiveMap["filename"].front() != '/') {directiveMap["filename"] = outputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getWriter(directiveMap, symTable, recordTable)->writeAll(*rel_1_float_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+}+public:+void loadAll(std::string inputDirectory = ".") override {+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"filename","./string_fact.facts"},{"name","string_fact"},{"operation","input"},{"types","{\"records\": {}, \"string_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"s:symbol\"]}}"}});+if (!inputDirectory.empty() && directiveMap["IO"] == "file" && directiveMap["filename"].front() != '/') {directiveMap["filename"] = inputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getReader(directiveMap, symTable, recordTable)->readAll(*rel_3_string_fact);+} catch (std::exception& e) {std::cerr << "Error loading data: " << e.what() << '\n';}+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"filename","./number_fact.facts"},{"name","number_fact"},{"operation","input"},{"types","{\"number_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"i:number\"]}, \"records\": {}}"}});+if (!inputDirectory.empty() && directiveMap["IO"] == "file" && directiveMap["filename"].front() != '/') {directiveMap["filename"] = inputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getReader(directiveMap, symTable, recordTable)->readAll(*rel_2_number_fact);+} catch (std::exception& e) {std::cerr << "Error loading data: " << e.what() << '\n';}+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"filename","./unsigned_fact.facts"},{"name","unsigned_fact"},{"operation","input"},{"types","{\"records\": {}, \"unsigned_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"u:unsigned\"]}}"}});+if (!inputDirectory.empty() && directiveMap["IO"] == "file" && directiveMap["filename"].front() != '/') {directiveMap["filename"] = inputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getReader(directiveMap, symTable, recordTable)->readAll(*rel_4_unsigned_fact);+} catch (std::exception& e) {std::cerr << "Error loading data: " << e.what() << '\n';}+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"filename","./float_fact.facts"},{"name","float_fact"},{"operation","input"},{"types","{\"float_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"f:float\"]}, \"records\": {}}"}});+if (!inputDirectory.empty() && directiveMap["IO"] == "file" && directiveMap["filename"].front() != '/') {directiveMap["filename"] = inputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getReader(directiveMap, symTable, recordTable)->readAll(*rel_1_float_fact);+} catch (std::exception& e) {std::cerr << "Error loading data: " << e.what() << '\n';}+}+public:+void dumpInputs(std::ostream& out = std::cout) override {+try {std::map<std::string, std::string> rwOperation;+rwOperation["IO"] = "stdout";+rwOperation["name"] = "string_fact";+rwOperation["types"] = "{\"string_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"s:symbol\"]}}";+IOSystem::getInstance().getWriter(rwOperation, symTable, recordTable)->writeAll(*rel_3_string_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+try {std::map<std::string, std::string> rwOperation;+rwOperation["IO"] = "stdout";+rwOperation["name"] = "number_fact";+rwOperation["types"] = "{\"number_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"i:number\"]}}";+IOSystem::getInstance().getWriter(rwOperation, symTable, recordTable)->writeAll(*rel_2_number_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+try {std::map<std::string, std::string> rwOperation;+rwOperation["IO"] = "stdout";+rwOperation["name"] = "unsigned_fact";+rwOperation["types"] = "{\"unsigned_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"u:unsigned\"]}}";+IOSystem::getInstance().getWriter(rwOperation, symTable, recordTable)->writeAll(*rel_4_unsigned_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+try {std::map<std::string, std::string> rwOperation;+rwOperation["IO"] = "stdout";+rwOperation["name"] = "float_fact";+rwOperation["types"] = "{\"float_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"f:float\"]}}";+IOSystem::getInstance().getWriter(rwOperation, symTable, recordTable)->writeAll(*rel_1_float_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+}+public:+void dumpOutputs(std::ostream& out = std::cout) override {+try {std::map<std::string, std::string> rwOperation;+rwOperation["IO"] = "stdout";+rwOperation["name"] = "string_fact";+rwOperation["types"] = "{\"string_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"s:symbol\"]}}";+IOSystem::getInstance().getWriter(rwOperation, symTable, recordTable)->writeAll(*rel_3_string_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+try {std::map<std::string, std::string> rwOperation;+rwOperation["IO"] = "stdout";+rwOperation["name"] = "unsigned_fact";+rwOperation["types"] = "{\"unsigned_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"u:unsigned\"]}}";+IOSystem::getInstance().getWriter(rwOperation, symTable, recordTable)->writeAll(*rel_4_unsigned_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+try {std::map<std::string, std::string> rwOperation;+rwOperation["IO"] = "stdout";+rwOperation["name"] = "number_fact";+rwOperation["types"] = "{\"number_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"i:number\"]}}";+IOSystem::getInstance().getWriter(rwOperation, symTable, recordTable)->writeAll(*rel_2_number_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+try {std::map<std::string, std::string> rwOperation;+rwOperation["IO"] = "stdout";+rwOperation["name"] = "float_fact";+rwOperation["types"] = "{\"float_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"f:float\"]}}";+IOSystem::getInstance().getWriter(rwOperation, symTable, recordTable)->writeAll(*rel_1_float_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+}+public:+SymbolTable& getSymbolTable() override {+return symTable;+}+void executeSubroutine(std::string name, const std::vector<RamDomain>& args, std::vector<RamDomain>& ret) override {+if (name == "stratum_0") {+subroutine_0(args, ret);+return;}+if (name == "stratum_1") {+subroutine_1(args, ret);+return;}+if (name == "stratum_2") {+subroutine_2(args, ret);+return;}+if (name == "stratum_3") {+subroutine_3(args, ret);+return;}+fatal("unknown subroutine");+}+void subroutine_0(const std::vector<RamDomain>& args, std::vector<RamDomain>& ret) {+if (performIO) {+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"filename","./string_fact.facts"},{"name","string_fact"},{"operation","input"},{"types","{\"records\": {}, \"string_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"s:symbol\"]}}"}});+if (!inputDirectory.empty() && directiveMap["filename"].front() != '/') {directiveMap["filename"] = inputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getReader(directiveMap, symTable, recordTable)->readAll(*rel_3_string_fact);+} catch (std::exception& e) {std::cerr << "Error loading data: " << e.what() << '\n';}+}+if (performIO) {+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"attributeNames","x"},{"filename","./string_fact.csv"},{"name","string_fact"},{"operation","output"},{"types","{\"records\": {}, \"string_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"s:symbol\"]}}"}});+if (!outputDirectory.empty() && directiveMap["filename"].front() != '/') {directiveMap["filename"] = outputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getWriter(directiveMap, symTable, recordTable)->writeAll(*rel_3_string_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+}+}+void subroutine_1(const std::vector<RamDomain>& args, std::vector<RamDomain>& ret) {+if (performIO) {+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"filename","./number_fact.facts"},{"name","number_fact"},{"operation","input"},{"types","{\"number_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"i:number\"]}, \"records\": {}}"}});+if (!inputDirectory.empty() && directiveMap["filename"].front() != '/') {directiveMap["filename"] = inputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getReader(directiveMap, symTable, recordTable)->readAll(*rel_2_number_fact);+} catch (std::exception& e) {std::cerr << "Error loading data: " << e.what() << '\n';}+}+if (performIO) {+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"attributeNames","x"},{"filename","./number_fact.csv"},{"name","number_fact"},{"operation","output"},{"types","{\"number_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"i:number\"]}, \"records\": {}}"}});+if (!outputDirectory.empty() && directiveMap["filename"].front() != '/') {directiveMap["filename"] = outputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getWriter(directiveMap, symTable, recordTable)->writeAll(*rel_2_number_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+}+}+void subroutine_2(const std::vector<RamDomain>& args, std::vector<RamDomain>& ret) {+if (performIO) {+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"filename","./unsigned_fact.facts"},{"name","unsigned_fact"},{"operation","input"},{"types","{\"records\": {}, \"unsigned_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"u:unsigned\"]}}"}});+if (!inputDirectory.empty() && directiveMap["filename"].front() != '/') {directiveMap["filename"] = inputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getReader(directiveMap, symTable, recordTable)->readAll(*rel_4_unsigned_fact);+} catch (std::exception& e) {std::cerr << "Error loading data: " << e.what() << '\n';}+}+if (performIO) {+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"attributeNames","x"},{"filename","./unsigned_fact.csv"},{"name","unsigned_fact"},{"operation","output"},{"types","{\"records\": {}, \"unsigned_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"u:unsigned\"]}}"}});+if (!outputDirectory.empty() && directiveMap["filename"].front() != '/') {directiveMap["filename"] = outputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getWriter(directiveMap, symTable, recordTable)->writeAll(*rel_4_unsigned_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+}+}+void subroutine_3(const std::vector<RamDomain>& args, std::vector<RamDomain>& ret) {+if (performIO) {+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"filename","./float_fact.facts"},{"name","float_fact"},{"operation","input"},{"types","{\"float_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"f:float\"]}, \"records\": {}}"}});+if (!inputDirectory.empty() && directiveMap["filename"].front() != '/') {directiveMap["filename"] = inputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getReader(directiveMap, symTable, recordTable)->readAll(*rel_1_float_fact);+} catch (std::exception& e) {std::cerr << "Error loading data: " << e.what() << '\n';}+}+if (performIO) {+try {std::map<std::string, std::string> directiveMap({{"IO","file"},{"attributeNames","x"},{"filename","./float_fact.csv"},{"name","float_fact"},{"operation","output"},{"types","{\"float_fact\": {\"arity\": 1, \"auxArity\": 0, \"types\": [\"f:float\"]}, \"records\": {}}"}});+if (!outputDirectory.empty() && directiveMap["filename"].front() != '/') {directiveMap["filename"] = outputDirectory + "/" + directiveMap["filename"];}+IOSystem::getInstance().getWriter(directiveMap, symTable, recordTable)->writeAll(*rel_1_float_fact);+} catch (std::exception& e) {std::cerr << e.what();exit(1);}+}+}+};+SouffleProgram *newInstance_round_trip(){return new Sf_round_trip;}+SymbolTable *getST_round_trip(SouffleProgram *p){return &reinterpret_cast<Sf_round_trip*>(p)->symTable;}++#ifdef __EMBEDDED_SOUFFLE__+class factory_Sf_round_trip: public souffle::ProgramFactory {+SouffleProgram *newInstance() {+return new Sf_round_trip();+};+public:+factory_Sf_round_trip() : ProgramFactory("round_trip"){}+};+extern "C" {+factory_Sf_round_trip __factory_Sf_round_trip_instance;+}+}+#else+}+int main(int argc, char** argv)+{+try{+souffle::CmdOptions opt(R"(round_trip.dl)",+R"(.)",+R"(.)",+false,+R"()",+1);+if (!opt.parse(argc,argv)) return 1;+souffle::Sf_round_trip obj;+#if defined(_OPENMP) +obj.setNumThreads(opt.getNumJobs());++#endif+obj.runAll(opt.getInputFileDir(), opt.getOutputFileDir());+return 0;+} catch(std::exception &e) { souffle::SignalHandler::instance()->error(e.what());}+}++#endif