DBFunctor 0.1.0.0 → 0.1.1.0

raw patch · 8 files changed

+1681/−325 lines, 8 filesdep +timePVP: major bump suggested

API removals or changes: PVP suggests a major version bump

Dependencies added: time

API changes (from Hackage documentation)

Files

• ChangeLog.md +35/−9
• DBFunctor.cabal +20/−17
• README.md +142/−95
• app/Main.hs +2/−0
• src/Etl/Julius.hs +332/−19
• src/RTable/Core.hs +798/−117
• src/RTable/Data/CSV.hs +40/−41
• test/DBFTests.hs +312/−27

@@ -1,9 +1,35 @@-# Changelog for DBFunctor
-
-### 0.1.0.0
- - Initial Version. Includes a full-working version of 
-	 - Julius: A type-level Embedded Domain Specific (EDSL) Language for ETL
-	 - all common Relational Algebra operations, 
-	 - the ETL Mapping and other typical ETL constructs and operations
-	 - operations applicable to all kinds of tabular data
-	 - In-memory, database-less data processing.
+# Changelog for DBFunctor++### 0.1.0.0+ - Initial Version. Includes a full-working version of +	 - Julius: A type-level Embedded Domain Specific (EDSL) Language for ETL+	 - all common Relational Algebra operations, +	 - the ETL Mapping and other typical ETL constructs and operations+	 - operations applicable to all kinds of tabular data+	 - In-memory, database-less data processing.+	 +### 0.1.1.0+ - Includes various enhancements (most notable is DML operations support) and fixes +	 - Issue #1: Implemented agg function string_agg (listagg in Oracle) and the corresponding Julius clause+	 - Issue #2: Implemented Julius Aggregate clauses: CountDist and CountStar+	 - Issue #6 DML Enhancements+	 	- Implement Update Julius Clause+	 	- Implement Insert Operation and corresponding Julius Clause (both single RTuple INSERT and INSERT INTO SELECT) +	 	- Implement Merge/Upsert operation and corresponding Julius clause+	 	- Implement semi-join operation and corresponding Julius clause+	 	- Implement anti-join operation and corresponding Julius clause+	 	- Implement Delete operation and corresponding Julius clause+	 - Issue #5 : Add support for UTCTime+	 - Solve the CSV orphan instances problem by defining CSV with newtype+	 - Fix problem with order by. I have noticed the following bug:+```Haskell+								>>> let t1 = RDate {rdate = "01/12/1990", dtformat = "DD/MM/YYYY"}+								>>> let t2 = RDate {rdate = "1/12/1991", dtformat = "DD/MM/YYYY"}+								>>>	compare t1 t2+								>>>	EQ+```+Fix: +- Redefine the RDataType Ord instance based on the compare function instead of the (<=) function.+- When comparing RDate types, convert them first to RTimeStamps and then compare these ones+- The previous point apply it also to the Eq instance for RDataType.+

@@ -1,11 +1,13 @@--- This file has been generated from package.yaml by hpack version 0.20.0.+cabal-version: 1.12++-- This file has been generated from package.yaml by hpack version 0.31.1. -- -- see: https://github.com/sol/hpack ----- hash: 99671845ea773b6269f7c818d876e85741000c789f156dfc6c6f5d513eec516e+-- hash: 655aa8afa24b4ad882dcf70f7a96929f9946f310d5fab43f215e9df82dab2dcf  name:           DBFunctor-version:        0.1.0.0+version:        0.1.1.0 synopsis:       DBFunctor - Functional Data Management =>  ETL/ELT Data Processing in Haskell description:    Please see the README on Github at https://github.com/nkarag/haskell-DBFunctor category:       ETL@@ -17,17 +19,22 @@ license:        BSD3 license-file:   LICENSE build-type:     Simple-cabal-version:  >= 1.10- extra-source-files:-    ChangeLog.md     README.md+    ChangeLog.md  source-repository head   type: git   location: https://github.com/nkarag/DBFunctor  library+  exposed-modules:+      Etl.Internal.Core+      Etl.Julius+      RTable.Core+      RTable.Data.CSV+  other-modules:+      Paths_DBFunctor   hs-source-dirs:       src   build-depends:@@ -40,20 +47,16 @@     , deepseq     , either     , text+    , time     , transformers     , unordered-containers     , vector-  exposed-modules:-      Etl.Internal.Core-      Etl.Julius-      RTable.Core-      RTable.Data.CSV-  other-modules:-      Paths_DBFunctor   default-language: Haskell2010  executable dbfunctor-example   main-is: Main.hs+  other-modules:+      Paths_DBFunctor   hs-source-dirs:       app   ghc-options: -threaded -rtsopts -with-rtsopts=-N -Wall@@ -68,16 +71,17 @@     , deepseq     , either     , text+    , time     , transformers     , unordered-containers     , vector-  other-modules:-      Paths_DBFunctor   default-language: Haskell2010  test-suite dbfunctor-test   type: exitcode-stdio-1.0   main-is: DBFTests.hs+  other-modules:+      Paths_DBFunctor   hs-source-dirs:       test   ghc-options: -threaded -rtsopts -with-rtsopts=-N -Wall@@ -92,9 +96,8 @@     , deepseq     , either     , text+    , time     , transformers     , unordered-containers     , vector-  other-modules:-      Paths_DBFunctor   default-language: Haskell2010

@@ -1,13 +1,13 @@ ![dbfunctor logo](./dbfunctor.png)
 # DBFunctor:  Functional Data Management
 ## ETL/ELT* Data Processing in Haskell
-**DBFunctor** is a [Haskell](https://haskell-lang.org/) library for *ETL/ELT[^1]* data processing of tabular data. What does this mean?
+**[DBFunctor](https://hackage.haskell.org/package/DBFunctor)** is a [Haskell](https://haskell-lang.org/) library for *ETL/ELT[^1]* data processing of tabular data. What does this mean?
 It simply means that whenever you have a ***data analysis*, *data preparation*, or *data transformation* task** and you want to do it with Haskell type-safe code, that you enjoy, love and trust so much, now you can! 
 ### Main Features
- 1. **Julius: An Embedded Domain Specific (EDSL) Language for ETL**
+ 1. **[Julius](https://hackage.haskell.org/package/DBFunctor-0.1.0.0/docs/Etl-Julius.html) DSL: A Type-Level Embedded Domain Specific  Language (EDSL) for ETL**
 Provides an intuitive type-level Embedded Domain Specific (EDSL) Language called *Julius* for expressing complex data flows (i.e., ETL flows)  but also for performing SQL-like data analysis. For more info check this [Julius tutorial](https://github.com/nkarag/haskell-DBFunctor/blob/master/doc/JULIUS-TUTORIAL.md).
- 2. **Supports all known relational operations**
-Julius supports all known relational operations (selection, projection, inner/outer join, grouping, ordering, aggregation, set operations etc.)
+ 2. **Supports all known relational algrebra operations**
+Julius supports all known relational algebra operations (selection, projection, inner/outer join, grouping, ordering, aggregation, set operations etc.)
  3. **Provides the ETL Mapping and other typical ETL constructs and operations**
 Julius implements typical ETL constructs such the *Column Mapping* and the *ETL Mapping*.
  4. **Applicable to all kinds of tabular data**
@@ -137,21 +137,29 @@  **2. Query the RTable**
 Once we have created an RTable, we can issue queries on it, or apply any type of data transformations. Note that due to immutability, each query or data transformation creates a new RTable.
  We will now issue the following query:
-We return all the rows, which correspond to some filter predicate - in particular all rows where the table_name starts with a 'B'. For this we use the Julius EDSL, in order to express the query and then with the function `juliusToRTable :: ETLMappingExpr -> RTable `, we evaluate the expression into an RTable.
+We return all the rows, which correspond to some filter predicate - in particular all rows where the `TABLE_NAME` includes some search string and the `LAST_ANALYZED` field is greater than an input date.
+
+For this we use the Julius EDSL, in order to express the query and then with the function 
+`runJulius`, we evaluate the expression into an RTable.
 ```haskell
-tabs_with_B = juliusToRTable $
-   EtlMapStart
-   :-> (EtlR $
-           ROpStart
-              -- apply a filter on RTable "src_DBTab" based on a predicate, expressed with a lambda expression
-           :. (Filter (From $ Tab src_DBTab) $                 
-                   FilterBy (\t ->	let fstChar = Data.Text.take 1 $ fromJust $ toText (t <!> "TABLE_NAME")
-						            in fstChar == (pack "B"))
-           )
-              -- A simple column projection applied on the Previous result
-           :. (Select ["OWNER", "TABLE_NAME"] $  From Previous)
-   )
+runJulius :: ETLMappingExpr -> IO RTable
 ```
+Here is the Julius expression that yield the desired results.
+```haskell
+julExpr srch dtstr rtab = 
+            EtlMapStart 
+                :-> (EtlR $
+                        ROpStart
+                        :.  (Filter (From $ Tab rtab) $
+                                FilterBy (\t -> case instrRText (RText srch) (t <!> "TABLE_NAME") of
+                                                    Just p  -> True
+                                                    Nothing -> False
+                                                &&
+                                                (t <!> "LAST_ANALYZED") >= (RTime $ toRTimestamp "DD/MM/YYYY" dtstr)
+                                         )
+                            )
+                    )
+```
 A Julius expression is a *data processing chain* consisting of various Relational Algebra operations `(EtlR $ ...)`  and/or column mappings `(EtlC $ ...)` connected together via the `:->` data constructor,  of the form (Julius expressions are read *from top-to-bottom  or from left-to-right*):
 ```haskell
 myJulExpression =
@@ -171,48 +179,69 @@ 	:-> (EtlC $ ...) -- This is Column Mapping 4
 	...
 ```
-In our example, the Julius expression consists only of two relational algebra operations: a `Filter` operation, which uses an RTuple  predicate of the form 	`RTuple -> Bool` to filter out RTuples (i.e., rows) that dont satisfy this predicate. The predicate is expressed as the lambda expression:
+In our example, the Julius expression consists only of a single relational algebra operation, namely a `Filter` operation, which uses an RTuple  predicate of the form 	`RTuple -> Bool` to filter out RTuples (i.e., rows) that dont satisfy this predicate. The predicate is expressed as the lambda expression:
 ```haskell
-FilterBy (\t ->	let fstChar = Data.Text.take 1 $ fromJust $ toText (t <!> "TABLE_NAME")
-				in fstChar == (pack "B"))
+FilterBy (\t -> case instrRText (RText srch) (t <!> "TABLE_NAME") of
+                                                    Just p  -> True
+                                                    Nothing -> False
+                                                &&
+                                                (t <!> "LAST_ANALYZED") >= (RTime $ toRTimestamp "DD/MM/YYYY" dtstr)
 ```
-The second relational operation is a simple Projection expressed with the node `(Select ["OWNER", "TABLE_NAME"] $ From Previous)`
-Finally, in order to print the result of the query on the screen, we use the `printfRTable :: RTupleFormat -> RTable -> IO()` function, which brings printf-like functionality into the printing of RTables
+We use the [instrRText](https://hackage.haskell.org/package/DBFunctor-0.1.0.0/docs/RTable-Core.html#g:16) function to find these table_name values that include the `srch` string. Also, we use the [toRTimestamp](https://hackage.haskell.org/package/DBFunctor-0.1.0.0/docs/RTable-Core.html#g:15) function, in order to turn the date string `dtstr` into an `RTimestamp` data type and compare it against the `LAST_ANALYZED` column,
+
+Finally, in order to print the result of the query on the screen, we use the 
+```haskell
+printfRTable :: RTupleFormat -> RTable -> IO()
+``` 
+function, which brings printf-like functionality into the printing of RTables
 And here is the output:
 ```
-$ stack exec -- dbfunctor-example
-These are the tables that start with a "B":
+$ stack exec -- exampleDBFunctor
 
--------------------------------------
-OWNER      TABLE_NAME                
-~~~~~      ~~~~~~~~~~                
-DBSNMP     BSLN_BASELINES            
-DBSNMP     BSLN_METRIC_DEFAULTS      
-DBSNMP     BSLN_STATISTICS           
-DBSNMP     BSLN_THRESHOLD_PARAMS     
-SYS        BOOTSTRAP$                
+Print all tables that incude a "search string" in their name and have been analyzed after a specific date
 
-5 rows returned
--------------------------------------
-```
-Here is the complete example.
-```haskell
-module Main where
+Give me the search string:
+FLOW
+Give me the date in "DD/MM/YYYY" format:
+01/04/2018
+---------------------------------------------------------------------------------------------------------------------------------
+OWNER           TABLE_NAME                        TABLESPACE_NAME     STATUS     NUM_ROWS     BLOCKS     LAST_ANALYZED
+~~~~~           ~~~~~~~~~~                        ~~~~~~~~~~~~~~~     ~~~~~~     ~~~~~~~~     ~~~~~~     ~~~~~~~~~~~~~
+APEX_040100     WWV_FLOW_ACTIVITY_LOG1$           SYSAUX              VALID      4052         155        04/04/2018 18:19:56
+APEX_040100     WWV_FLOW_ACTIVITY_LOG2$           SYSAUX              VALID      1771         92         16/04/2018 17:33:16
+APEX_040100     WWV_FLOW_ACTIVITY_LOG_NUMBER$     SYSAUX              VALID      1            3          10/04/2018 16:09:25
+APEX_040100     WWV_FLOW_COMPANIES                SYSAUX              VALID      10           3          16/04/2018 17:33:13
+APEX_040100     WWV_FLOW_DATA                     SYSAUX              VALID      109          155        16/04/2018 16:06:38
+APEX_040100     WWV_FLOW_DEBUG_MESSAGES2          SYSAUX              VALID      0            0          10/04/2018 16:09:25
+APEX_040100     WWV_FLOW_FND_USER                 SYSAUX              VALID      50           3          05/04/2018 18:09:23
+APEX_040100     WWV_FLOW_PAGE_CACHE               SYSAUX              VALID      22           3          05/04/2018 18:35:18
+APEX_040100     WWV_FLOW_SESSIONS$                SYSAUX              VALID      182          26         16/04/2018 16:07:13
+APEX_040100     WWV_FLOW_USER_ACCESS_LOG1$        SYSAUX              VALID      127          5          11/04/2018 18:27:17
+APEX_040100     WWV_FLOW_USER_ACCESS_LOG2$        SYSAUX              VALID      39           5          16/04/2018 17:30:59
+APEX_040100     WWV_FLOW_USER_ACCESS_LOG_NUM$     SYSAUX              VALID      1            3          12/04/2018 16:30:05
+APEX_040100     WWV_FLOW_WORKSHEET_CONDITIONS     SYSAUX              VALID      501          18         16/04/2018 17:33:03
+APEX_040100     WWV_FLOW_WORKSHEET_GROUP_BY       SYSAUX              VALID      22           3          03/04/2018 17:44:07
+APEX_040100     WWV_FLOW_WORKSHEET_RPTS           SYSAUX              VALID      505          16         16/04/2018 17:33:01
+FLOWS_FILES     WWV_FLOW_FILE_OBJECTS$            SYSAUX              VALID      302          16         13/04/2018 18:00:05
 
-import  RTable.Core         (RTableMData ,ColumnDType (..) ,createRTableMData, printfRTable, genRTupleFormat, genDefaultColFormatMap, toText, (<!>))
-import  RTable.Data.CSV     (CSV, readCSV, toRTable)
-import  Etl.Julius          
 
-import Data.Text            (take, pack)
-import Data.Maybe           (fromJust)
+16 rows returned
+---------------------------------------------------------------------------------------------------------------------------------
+```
+**Here is the complete example**.
+```haskell
+{-# LANGUAGE OverloadedStrings #-}
 
+module Main where
 
---  Define Source Schema (i.e., a set of tables)
+import  Etl.Julius  
+import  RTable.Data.CSV     (CSV, readCSV, toRTable, writeCSV)
+import  Data.Text.IO as T   (getLine)
 
--- | This is the basic source table
--- It includes the tables of an imaginary database
+-- This is the input source table metadata
+-- It includes the tables stored in an imaginary database
 src_DBTab_MData :: RTableMData
-src_DBTab_MData =
+src_DBTab_MData = 
     createRTableMData   (   "sourceTab"  -- table name
                             ,[  ("OWNER", Varchar)                                      -- Owner of the table
                                 ,("TABLE_NAME", Varchar)                                -- Name of the table
@@ -220,75 +249,93 @@                                 ,("STATUS",Varchar)                                     -- Status of the table object (VALID/IVALID)
                                 ,("NUM_ROWS", Integer)                                  -- Number of rows in the table
                                 ,("BLOCKS", Integer)                                    -- Number of Blocks allocated for this table
-                                ,("LAST_ANALYZED", Timestamp "MM/DD/YYYY HH24:MI:SS")   -- Timestamp of the last time the table was analyzed (i.e., gathered statistics)
+                                ,("LAST_ANALYZED", Timestamp "MM/DD/YYYY HH24:MI:SS")   -- Timestamp of the last time the table was analyzed (i.e., gathered statistics) 
                             ]
                         )
                         ["OWNER", "TABLE_NAME"] -- primary key
                         [] -- (alternative) unique keys
 
--- | Define Target Schema (i.e., a set of tables)
-
+-- Result RTable metadata
+result_tab_MData :: RTableMData
+result_tab_MData = 
+    createRTableMData   (   "resultTab"  -- table name
+                            ,[  ("OWNER", Varchar)                                      -- Owner of the table
+                                ,("TABLE_NAME", Varchar)                                -- Name of the table
+                                ,("LAST_ANALYZED", Timestamp "MM/DD/YYYY HH24:MI:SS")   -- Timestamp of the last time the table was analyzed (i.e., gathered statistics) 
+                            ]
+                        )
+                        ["OWNER", "TABLE_NAME"] -- primary key
+                        [] -- (alternative) unique keys
 
 main :: IO ()
 main = do
-    -- read source csv file
+     -- read source csv file
     srcCSV <- readCSV "./app/test-data.csv"
 
-    let
-        -- create source RTable from source csv
-        src_DBTab = toRTable src_DBTab_MData srcCSV      
-
-        -- select all tables starting with a B
-        tabs_with_B = juliusToRTable $
-                EtlMapStart
-                :-> (EtlR $
-                        ROpStart                            
-                        :. (Filter (From $ Tab src_DBTab) $                 
-                                FilterBy (\t ->  let fstChar = Data.Text.take 1 $ fromJust $ toText (t <!> "TABLE_NAME") in fstChar == (pack "B"))
-                        )
-                        :. (Select ["OWNER", "TABLE_NAME"] $
-                            From Previous
-                        )
-                )
+    putStrLn "\nPrint all tables that incude a \"search string\" in their name and have been analyzed after a specific date\n"
+    putStrLn "Give me the search string: "
+    search <- T.getLine
 
-    putStrLn "\nThese are the tables that start with a \"B\":\n"
+    putStrLn "Give me the date in \"DD/MM/YYYY\" format: "
+    datestr <- Prelude.getLine
+        
 
-    -- print source RTable first 100 rows
+    -- print source RTable first n rows
+    -- RTable A
+    resultRTab <- runJulius $ julExpr search datestr $ toRTable src_DBTab_MData srcCSV 
     printfRTable (  
-                    -- this is the equivalent when pinting on the screen to a list of columns in a SELECT clause in SQL
-                    genRTupleFormat ["OWNER", "TABLE_NAME"] genDefaultColFormatMap
-                 ) $ tabs_with_B
-```
+                    -- this is the equivalent when printing on the screen a list of columns, defined in a SELECT clause in SQL
+                    genRTupleFormat ["OWNER", "TABLE_NAME", "TABLESPACE_NAME", "STATUS", "NUM_ROWS", "BLOCKS", "LAST_ANALYZED"] genDefaultColFormatMap
+                 ) $ resultRTab  
 
-### Julius Tutorial 
+    -- save result to a CSV file
+    writeCSV "./app/result-data.csv" $ 
+                    fromRTable result_tab_MData resultRTab
+    where
+        julExpr srch dtstr rtab = 
+            EtlMapStart 
+                :-> (EtlR $
+                        ROpStart
+                        :.  (Filter (From $ Tab rtab) $
+                                FilterBy (\t -> case instrRText (RText srch) (t <!> "TABLE_NAME") of
+                                                    Just p  -> True
+                                                    Nothing -> False
+                                                &&
+                                                (t <!> "LAST_ANALYZED") >= (RTime $ toRTimestamp "DD/MM/YYYY" dtstr)
+                                         )
+                            )
+                    )
+```
+### Julius DSL Tutorial 
 We have written [a Julius tutorial](https://github.com/nkarag/haskell-DBFunctor/blob/master/doc/JULIUS-TUTORIAL.md) to help you get started with Julius DSL. 
 <a  name="howtorun"></a> 
 ### How to run
-Download or clone DBFunctor in a new directory, 
-```
-$ git clone https://github.com/nkarag/haskell-DBFunctor
-$ cd haskell-DBFunctor/
-```
-then run 
+#### 1. Install Stack
+See [this](https://docs.haskellstack.org/en/stable/GUIDE/) guide for help. If you have stack already installed, then we suggest you run a `stack upgrade`, in order to update it to the latest version and avoid any error messages due to bugs.
+Then run a `stack update`, in order to update the package index.
+#### 2. Initiate a project
+`$ stack new myDBFunctorProject`
+#### 3. Start coding with DBFunctor
+Don't forget:
+
+ - to `import Etl.Julius` module
+ - to use GHC extension `{-# LANGUAGE OverloadedStrings #-}`, since DBFunctor uses `Text` in all of its basic data types, this extension is necessary if you want to assign string literal values to an `RDataType` (the type of a column in an `RTable`)
+#### 4. Declare dependency with DBFunctor package
+Edit your package.yaml (or project.cabal) file and add the dependency to the DBFunctor package
 ```
-$ stack build --haddock
+dependencies:
+- DBFunctor
 ```
-in order to build the code and generate the documentation with the [stack](https://docs.haskellstack.org/en/stable/README/) tool.
-In order, to use it in you own haskell app, you only need to import the Julius module (you dont have to import RTable.Core, because it is exported by Julius)
-```Haskell
-import Etl.Julius
+Also, don't forget to add to your stack.yaml file the line:
 ```
-Finally, for a successful build of your app, you must direct stack to treat it as a local package (Soon DBFunctor will be added to Hackage and you will not need to use it as a local package.). 
-So you have to include it in your stack.yaml file, as a local package that you want to link your code to. 
+   extra-deps:
+    - DBFunctor-0.1.0.0
 ```
-packages:
-- location: .
-- location: <path where DBFunctor package has been cloned>
-  extra-dep: true
+#### 5. Build and run you project
 ```
-And of course, you must not forget to add the dependency of your app to the DBFunctor package in your .cabal file
+$ stack build
 ```
-  build-depends:      ...
-                      , DBFunctor
-
+Run
+``` 
+stack exec -- myDBFunctorProject-exe
 ```

@@ -1,5 +1,7 @@ {-# LANGUAGE OverloadedStrings #-}
 
+-- :set -XOverloadedStrings
+
 module Main where
 
 -- import  RTable.Core         (RTableMData ,ColumnDType (..) ,createRTableMData, restrictNrows, printfRTable, genRTupleFormat, genDefaultColFormatMap, toText, (<!>))

@@ -471,8 +471,9 @@ 
 -}
 
-{-# LANGUAGE OverloadedStrings #-}
+{-# LANGUAGE OverloadedStrings      #-}
 -- :set -XOverloadedStrings
+
 --{-# LANGUAGE OverloadedRecordFields #-}
 --{-# LANGUAGE  DuplicateRecordFields #-}
 
@@ -493,6 +494,8 @@     ,TabExpr (..)          
     ,RemoveSrcCol (..)    
     ,ByPred (..)    
+    ,ByDelPred (..)
+    ,SetColumns (..)
     -- ** The Relational Operation Clause
     ,ROpExpr(..)    
     ,RelationalOp (..)    
@@ -506,6 +509,15 @@     ,TabExprJoin (..)    
     ,ByGenUnaryOperation (..)
     ,ByGenBinaryOperation (..)
+    ,IntoClause (..)
+    ,InsertSource (..)
+    ,ValuesClause (..)
+    ,TabSource (..)
+    ,MergeInto (..)
+    ,MergeSource (..)
+    ,MergeMatchCondition (..)
+    ,WhenMatched (..)
+    ,UpdateColumns (..)
     -- * Julius Expression Evaluation
     ,evalJulius
     ,juliusToRTable
@@ -594,8 +606,11 @@ data RemoveSrcCol = RemoveSrc | DontRemoveSrc
 
 -- | An 'RTuple' predicate clause.
-data ByPred = FilterBy RPredicate
+data ByPred = FilterBy RPredicate 
 
+-- | Predicate for Deletion Operation
+data ByDelPred = Where RPredicate
+
 -- | A Relational Operation Expression ('ROpExpr') is a sequence of one or more Relational Algebra Operations applied on a input 'RTable'.
 -- It is a sub-expression within a Julius Expression ('ETLMappingExpr') and we use it whenever we want to apply relational algebra operations on an RTable
 -- (which might be the result of previous operations in a Julius Expression). A Julius Expression ('ETLMappingExpr') can contain an arbitrary number of
@@ -636,14 +651,150 @@         |   LJoin TabLiteral TabExpr TabExprJoin            -- ^ Left Join clause, based on an arbitrary join predicate function - not just equi-join - ('RJoinPredicate')
         |   RJoin TabLiteral TabExpr TabExprJoin            -- ^ Right Join clause, based on an arbitrary join predicate function - not just equi-join - ('RJoinPredicate')
         |   FOJoin TabLiteral TabExpr TabExprJoin           -- ^ Full Outer Join clause, based on an arbitrary join predicate function - not just equi-join - ('RJoinPredicate')
+        |   SemiJoin TabLiteral TabExpr TabExprJoin         
+            -- ^ Implements the semi-Join operation between two RTables (any type of join predicate is allowed)
+            -- It returns the 'RTuple's from the left 'RTable' that match with the right 'RTable'.
+            -- Note that if an 'RTuple' from the left 'RTable' matches more than one 'RTuple's from the right 'RTable'
+            -- the semi join operation will return only a single 'RTuple'.    
+        |   SemiJoinP TabExpr TabLiteral TabExprJoin        --  ^ This is a semi-Join operation to be used when the left table must be the 'Previous' value. 
+        |   AntiJoin TabLiteral TabExpr TabExprJoin         
+            -- ^ Implements the anti-Join operation between two RTables (any type of join predicate is allowed)
+            -- It returns the 'RTuple's from the left 'RTable' that DONT match with the right 'RTable'.
+        |   AntiJoinP TabExpr TabLiteral TabExprJoin        --  ^ This is a anti-Join operation to be used when the left table must be the 'Previous' value. 
+ 
         |   TabLiteral `Intersect` TabExpr                  -- ^ Intersection clause
-        |   TabLiteral `Union` TabExpr                      -- ^ Union clause
+        |   TabLiteral `Union` TabExpr                      -- ^ Union clause. Note this operation eliminates dublicate 'RTuples'
+        |   TabLiteral `UnionAll` TabExpr                   -- ^ Union All clause. It is a Union operation without dublicate 'RTuple' elimination.
         |   TabLiteral `Minus` TabExpr                      -- ^ Minus clause (set Difference operation)
         |   TabExpr `MinusP` TabLiteral                     --  ^ This is a Minus operation to be used when the left table must be the 'Previous' value.
         |   GenUnaryOp OnRTable ByGenUnaryOperation         -- ^ This is a generic unary operation on a RTable ('UnaryRTableOperation'). It is used to define an arbitrary unary operation on an 'RTable'
         |   GenBinaryOp TabLiteral TabExpr ByGenBinaryOperation -- ^ This is a generic binary operation on a RTable ('BinaryRTableOperation'). It is used to define an arbitrary binary operation on an 'RTable'
         |   OrderBy [(ColumnName, OrderingSpec)] FromRTable     -- ^ Order By clause.
+        
+        {--  DML Section  -}
+        |   Delete FromRTable ByDelPred     
+            -- ^ Delete operation. Deletes the 'RTuple's from an 'RTable' based on an 'RPredicate'.
+            -- Please note that this is an __immutable__ implementation of an 'RTable' update. This simply means that
+            -- the delete operation returns a new 'RTable'. So, the original 'RTable' remains unchanged and no deletion in-place
+            -- takes place whatsoever.
+            -- Moreover, if we have multiple threads deleting an 'RTable', due to immutability, each thread \"sees\" its own copy of
+            -- the 'RTable' and thus there is no need for locking the deleted 'RTuple's, as happens in a common RDBMS.
 
+        |   Update TabExpr SetColumns ByPred
+            -- ^ Update an 'RTable'. 
+            -- Please note that this is an __immutable__ implementation of an 'RTable' update. This simply means that
+            -- the update operation returns a new 'RTable' that includes all the 'RTuple's of the original 'RTable', both the ones
+            -- that have been updated and the others that have not. So, the original 'RTable' remains unchanged and no update in-place
+            -- takes place whatsoever.
+            -- Moreover, if we have multiple threads updating an 'RTable', due to immutability, each thread \"sees\" its own copy of
+            -- the 'RTable' and thus there is no need for locking the updated 'RTuple's, as happens in a common RDBMS.
+
+        |   Insert IntoClause
+           -- ^ Insert Operation. It can insert into an 'RTable' a single 'RTuple' or a whole 'RTable'. The latter is the equivalent
+            -- of an @INSERT INTO SELECT@ clause in SQL. Since, an 'RTable' can be the result of a Julius expression (playing the
+            -- role of a subquery within the Insert clause, in this case).
+            -- Please note that this is an __immutable__ implementation of an 'RTable' insert.
+            -- This simply means that the insert operation returns a new 'RTable' and does not
+            -- affect the original 'RTable'. 
+            -- Also note that the source and target 'RTable's should have the same structure.
+            -- By \"structure\", we mean that the 'ColumnName's and the corresponding data types must match. Essentially what we record in the 'ColumnInfo'
+            -- must be the same for the two 'RTable's. Otherwise a 'ConflictingRTableStructures' exception will be thrown.
+
+        |   Upsert MergeInto                                
+            -- ^ Upsert (Update+Insert, aka Merge) Operation. We provide a source 'RTable' and a matching condition ('RUpsertPredicate') to the 'RTuple's 
+            -- of the target 'RTable'. An 'RTuple' from the target 'RTable' might match to a single only 'RTuple' in the source 'RTable', or not match at all. 
+            -- If it is matched to more than one 'RTuple's then an exception ('UniquenessViolationInUpsert')is thrown. 
+            -- When an 'RTuple' from the target 'RTable' is matched to a source 'RTuple', then the corresponding columns of the target 'RTuple' are updated
+            -- with the new values provided in the source 'RTuple'. This takes place for the target 'RTuple's that match but also that satisfy the input 
+            -- 'RPredicate'. Thus we can restrict further with a filter the 'RTuple's of the target 'RTable' where the update will take place.
+            -- Finally, the source 'RTuple's that did not match to the target 'RTable', are inserted (appended) to the target 'RTable'
+            --
+            -- Please note that this is an __immutable__ implementation of an 'RTable' upsert.
+            -- This simply means that the upsert operation returns a new 'RTable' and does not
+            -- affect the original 'RTable'.
+            -- Also note that the source and target 'RTable's should have the same structure.
+            -- By \"structure\", we mean that the 'ColumnName's and the corresponding data types must match. Essentially what we record in the 'ColumnInfo'
+            -- must be the same for the two 'RTable's. Otherwise a 'ConflictingRTableStructures' exception will be thrown.
+            --
+            -- @
+            --  An Example:
+            --  Source RTable: srcTab = 
+            --      Id  |   Msg         | Other
+            --      ----|---------------|-------
+            --      1   |   "updated"   |"a"    
+            --      2   |   "world2"    |"a"    
+            --      3   |   "inserted"  |"a"    
+            --
+            --  Target RTable: trgTab = 
+            --      Id  |   Msg         | Other
+            --      ----|---------------|-------
+            --      1   |   "hello1"    |"b"    
+            --      2   |   "world1"    |"b"    
+            --      4   |   "old"       |"b"    
+            --      5   |   "hello"     |"b"    
+            --            
+            --  juliusToRTable $
+            --      EtlMapStart
+            --          :-> (EtlR $
+            --                  ROpStart
+            --                  :.(Upsert $ 
+            --                      MergeInto (Tab trgTab) $
+            --                          Using (TabSrc srcTab) $
+            --                              MergeOn (RUpsertPredicate [\"Id\"] (\\t1 t2 -> t1 \<!\> \"Id\" == t2 \<!\> \"Id\")) $  -- merge condition: srcTab.Id == trgTab.Id
+            --                                  WhenMatchedThen $
+            --                                      UpdateCols [\"Msg\"] $
+            --                                          FilterBy (\\t ->    let
+            --                                                                msg = case toText (t \<!\> \"Msg\") of
+            --                                                                  Just t -> t
+            --                                                                  Nothing -> pack ""
+            --                                                              in (take 5 msg) == (pack "hello")
+            --                                                   )  -- Msg like "hello%"
+            --                  )
+            --              )
+            --
+            --  Result RTable: 
+            --      Id  |   Msg         | Other
+            --      ----|---------------|-------
+            --      1   |   "updated"   |"b"   -- Updated RTuple. Note that only column \"Msg\" has been overwritten, as per the UpdateCols subclause
+            --      2   |   "world1"    |"b"   -- Not affected due to FilterBy predicate
+            --      3   |   "inserted"  |"a"   -- Inserted RTuple
+            --      4   |   "old"       |"b"   -- Not affected due to MergeOn condition
+            --      5   |   "hello"     |"b"   -- Not affected due to MergeOn condition  
+            -- @
+            --
+
+
+-- | Insert Into subclause
+data IntoClause = Into TabExpr InsertSource
+
+-- | Subclause on 'Insert' clause. Defines the source of the insert operation.
+-- The @Values@ branch is used for inserting a singl 'RTuple', while the @RTuples@ branch
+-- is used for inserting a whole 'RTable', typically derived as the result of a Julius expression.
+-- The former is similar in concept with an @INSERT INTO VALUES@ SQL clause, and the latter is similar 
+-- in concept with an @INSERT INTO SELECT@ SQL clause.
+data InsertSource = Values ValuesClause | RTuples TabSource
+
+-- | Merge Into subclause
+data MergeInto = MergeInto TabExpr MergeSource
+
+-- | Upsert source subclause (Using clause in @SQL@)
+data MergeSource = Using TabSource MergeMatchCondition
+
+-- | Upsert matching condition subclause
+data MergeMatchCondition =  MergeOn RUpsertPredicate WhenMatched
+
+-- | When Matched subclause of Upsert
+data WhenMatched = WhenMatchedThen UpdateColumns
+
+-- | Update columns subclause of Upsert
+data UpdateColumns = UpdateCols [ColumnName] ByPred 
+
+-- | This subclause refers to the source 'RTable' that will feed an 'Insert' operation
+data TabSource = TabSrc RTable 
+
+-- | Subclause on 'Insert' clause. Defines the source 'RTuple' of the insert operation.
+type ValuesClause = [(ColumnName, RDataType)]
+
 -- | It is used to define an arbitrary unary operation on an 'RTable'
 data ByGenUnaryOperation = ByUnaryOp UnaryRTableOperation         
 
@@ -672,14 +823,16 @@ -- | These are the available aggregate operation clauses
 data AggOp = 
                 Sum ColumnName AsColumn      
-            |   Count ColumnName AsColumn    -- ^ Count aggregation (no distinct)
-            -- |   CountDist ColumnName AsColumn    -- ^ Count distinct aggregation. Returns the distinct number of values for this column.
-            -- |   CountStar AsColumn              -- ^ Returns the number of 'RTuple's in the 'RTable' (i.e., @count(*)@ in SQL)           
+            |   Count ColumnName AsColumn     -- ^ Count aggregation (no distinct)
+            |   CountDist ColumnName AsColumn -- ^ Count distinct aggregation (i.e., @count(distinct col)@ in SQL). Returns the distinct number of values for this column.
+            |   CountStar AsColumn            -- ^ Returns the number of 'RTuple's in the 'RTable' (i.e., @count(*)@ in SQL)           
             |   Min ColumnName AsColumn
             |   Max ColumnName AsColumn
             |   Avg ColumnName AsColumn      -- ^ Average aggregation
+            |   StrAgg ColumnName AsColumn Delimiter  -- ^ String aggregation
             |   GenAgg ColumnName AsColumn AggBy    -- ^ A custom aggregate operation
 
+
 -- | Julius Clause to provide a custom aggregation function
 data AggBy = AggBy AggFunction
 
@@ -690,6 +843,9 @@ -- | A grouping predicate clause. It defines an arbitrary function ('RGroupPRedicate'), which drives when two 'RTuple's should belong in the same group.
 data GroupOnPred = GroupOn RGroupPredicate
 
+-- | The Set sub-clause of an 'Update' 'RTable' clause. It specifies each column to be updated along with the new value.
+data SetColumns = Set [(ColumnName, RDataType)]
+
 -----------------------
 -- Example:
 -----------------------
@@ -1072,7 +1228,7 @@ -- | Evaluates (parses) a Relational Operation Expression of the form 
 --
 -- @
---  ROp :. ROp :. ... :. ROpStart
+--  ROpStart :. ROp  ... :. ROp 
 -- @
 --
 -- and produces the corresponding ROperation data type.
@@ -1208,6 +1364,62 @@                     RCombinedOp {rcombOp = prevfunc} -> (RCombinedOp {rcombOp = currfunc . prevfunc}, prevTXEleft, EmptyTab)
                     -- in this case the current Operation is the last one (the previous is just empty and must be ignored)
                     ROperationEmpty ->  (RCombinedOp {rcombOp = currfunc}, TXE tabExpr, EmptyTab)
+
+        SemiJoin (TabL tabl) tabExpr (JoinOn joinPred) ->
+            let -- create current function to be included in a RCombinedOp operation 
+                currfunc :: UnaryRTableOperation
+                currfunc = sJ joinPred tabl -- this returns a RTable -> RTable function
+                -- get previous RCombinedOp operation and table expressions
+                (prevOperation, prevTXEleft, prevTXEright) = evalROpExpr restExpression                                
+
+            -- the current RCombinedOp is porduced by composing the current function with the previous function
+            in case prevOperation of
+                    -- in this case the previous operation is a valid non-empty operation and must be composed with the current one
+                    RCombinedOp {rcombOp = prevfunc} -> (RCombinedOp {rcombOp = currfunc . prevfunc}, prevTXEleft, EmptyTab)
+                    -- in this case the current Operation is the last one (the previous is just empty and must be ignored)
+                    ROperationEmpty ->  (RCombinedOp {rcombOp = currfunc}, TXE tabExpr, EmptyTab)
+
+        SemiJoinP tabExpr (TabL tabl) (JoinOn joinPred) ->
+            let -- create current function to be included in a RCombinedOp operation 
+                currfunc :: UnaryRTableOperation
+                currfunc = (flip (sJ joinPred)) tabl -- -- this returns a RTable -> RTable function, note that flip ensures that left table will be the second argument in the (sJ joinPred) function
+                -- get previous RCombinedOp operation and table expressions
+                (prevOperation, prevTXEleft, prevTXEright) = evalROpExpr restExpression                                
+
+            -- the current RCombinedOp is porduced by composing the current function with the previous function
+            in case prevOperation of
+                    -- in this case the previous operation is a valid non-empty operation and must be composed with the current one
+                    RCombinedOp {rcombOp = prevfunc} -> (RCombinedOp {rcombOp = currfunc . prevfunc}, prevTXEleft, EmptyTab)
+                    -- in this case the current Operation is the last one (the previous is just empty and must be ignored)
+                    ROperationEmpty ->  (RCombinedOp {rcombOp = currfunc}, TXE tabExpr, EmptyTab)
+
+        AntiJoin (TabL tabl) tabExpr (JoinOn joinPred) ->
+            let -- create current function to be included in a RCombinedOp operation 
+                currfunc :: UnaryRTableOperation
+                currfunc = aJ joinPred tabl -- this returns a RTable -> RTable function
+                -- get previous RCombinedOp operation and table expressions
+                (prevOperation, prevTXEleft, prevTXEright) = evalROpExpr restExpression                                
+
+            -- the current RCombinedOp is porduced by composing the current function with the previous function
+            in case prevOperation of
+                    -- in this case the previous operation is a valid non-empty operation and must be composed with the current one
+                    RCombinedOp {rcombOp = prevfunc} -> (RCombinedOp {rcombOp = currfunc . prevfunc}, prevTXEleft, EmptyTab)
+                    -- in this case the current Operation is the last one (the previous is just empty and must be ignored)
+                    ROperationEmpty ->  (RCombinedOp {rcombOp = currfunc}, TXE tabExpr, EmptyTab)
+
+        AntiJoinP tabExpr (TabL tabl) (JoinOn joinPred) ->
+            let -- create current function to be included in a RCombinedOp operation 
+                currfunc :: UnaryRTableOperation
+                currfunc = (flip (aJ joinPred)) tabl -- -- this returns a RTable -> RTable function, note that flip ensures that left table will be the second argument in the (aJ joinPred) function
+                -- get previous RCombinedOp operation and table expressions
+                (prevOperation, prevTXEleft, prevTXEright) = evalROpExpr restExpression                                
+
+            -- the current RCombinedOp is porduced by composing the current function with the previous function
+            in case prevOperation of
+                    -- in this case the previous operation is a valid non-empty operation and must be composed with the current one
+                    RCombinedOp {rcombOp = prevfunc} -> (RCombinedOp {rcombOp = currfunc . prevfunc}, prevTXEleft, EmptyTab)
+                    -- in this case the current Operation is the last one (the previous is just empty and must be ignored)
+                    ROperationEmpty ->  (RCombinedOp {rcombOp = currfunc}, TXE tabExpr, EmptyTab)
         
         Intersect (TabL tabl) tabExpr ->
             let -- create current function to be included in a RCombinedOp operation 
@@ -1237,6 +1449,20 @@                     -- in this case the current Operation is the last one (the previous is just empty and must be ignored)
                     ROperationEmpty ->  (RCombinedOp {rcombOp = currfunc}, TXE tabExpr, EmptyTab)
 
+        UnionAll (TabL tabl) tabExpr ->
+            let -- create current function to be included in a RCombinedOp operation 
+                currfunc :: UnaryRTableOperation
+                currfunc = runUnionAll tabl -- this returns a RTable -> RTable function
+                -- get previous RCombinedOp operation and table expressions
+                (prevOperation, prevTXEleft, prevTXEright) = evalROpExpr restExpression                                
+
+            -- the current RCombinedOp is porduced by composing the current function with the previous function
+            in case prevOperation of
+                    -- in this case the previous operation is a valid non-empty operation and must be composed with the current one
+                    RCombinedOp {rcombOp = prevfunc} -> (RCombinedOp {rcombOp = currfunc . prevfunc}, prevTXEleft, EmptyTab)
+                    -- in this case the current Operation is the last one (the previous is just empty and must be ignored)
+                    ROperationEmpty ->  (RCombinedOp {rcombOp = currfunc}, TXE tabExpr, EmptyTab)
+
         Minus (TabL tabl) tabExpr ->
             let -- create current function to be included in a RCombinedOp operation 
                 currfunc :: UnaryRTableOperation
@@ -1296,21 +1522,109 @@                     -- in this case the current Operation is the last one (the previous is just empty and must be ignored)
                     ROperationEmpty ->  (RCombinedOp {rcombOp = currfunc}, TXE tabExpr, EmptyTab)
 
+        -- this the RTable Delete operation
+        Delete (From tabExpr) (Where deletePred) ->
+            let -- create current function to be included in a RCombinedOp operation 
+                currfunc :: UnaryRTableOperation
+                currfunc = deleteRTab deletePred -- this returns a RTable -> RTable function
+                -- get previous RCombinedOp operation and table expressions
+                (prevOperation, prevTXEleft, prevTXEright) = evalROpExpr restExpression
 
+            -- the current RCombinedOp is produced by composing the current function with the previous function
+            in case prevOperation of
+                    -- in this case the previous operation is a valid non-empty operation and must be composed with the current one
+                    RCombinedOp {rcombOp = prevfunc} -> (RCombinedOp {rcombOp = currfunc . prevfunc}, prevTXEleft, EmptyTab)
+                    -- in this case the current Operation is the last one (the previous is just empty and must be ignored)
+                    ROperationEmpty ->  (RCombinedOp {rcombOp = currfunc}, TXE tabExpr, EmptyTab)                    
+
+
+        -- this is the RTable update operation
+        Update tabExpr (Set colValuePairs) (FilterBy filterPred) ->
+            let -- create current function to be included in a RCombinedOp operation 
+                currfunc :: UnaryRTableOperation
+                currfunc = updateRTab colValuePairs filterPred -- this returns an RTable -> RTable function
+                -- get previous RCombinedOp operation and table expressions
+                (prevOperation, prevTXEleft, prevTXEright) = evalROpExpr restExpression
+
+            -- the current RCombinedOp is produced by composing the current function with the previous function
+            in case prevOperation of
+                    -- in this case the previous operation is a valid non-empty operation and must be composed with the current one
+                    RCombinedOp {rcombOp = prevfunc} -> (RCombinedOp {rcombOp = currfunc . prevfunc}, prevTXEleft, EmptyTab)
+                    -- in this case the current Operation is the last one (the previous is just empty and must be ignored)
+                    ROperationEmpty ->  (RCombinedOp {rcombOp = currfunc}, TXE tabExpr, EmptyTab)                    
+
+        -- this is the RTable insert values operation (inserts a single RTuple)
+        Insert (Into tabExpr (Values colValuePairs)) ->
+            let -- create current function to be included in a RCombinedOp operation 
+                currfunc :: UnaryRTableOperation
+                currfunc = insertAppendRTab (createRTuple colValuePairs) -- this returns an RTable -> RTable function
+                -- get previous RCombinedOp operation and table expressions
+                (prevOperation, prevTXEleft, prevTXEright) = evalROpExpr restExpression
+
+            -- the current RCombinedOp is produced by composing the current function with the previous function
+            in case prevOperation of
+                    -- in this case the previous operation is a valid non-empty operation and must be composed with the current one
+                    RCombinedOp {rcombOp = prevfunc} -> (RCombinedOp {rcombOp = currfunc . prevfunc}, prevTXEleft, EmptyTab)
+                    -- in this case the current Operation is the last one (the previous is just empty and must be ignored)
+                    ROperationEmpty ->  (RCombinedOp {rcombOp = currfunc}, TXE tabExpr, EmptyTab)                    
+
+        -- this is the RTable insert rtuples operation (inserts a whole rtable)
+        Insert (Into tabExpr (RTuples (TabSrc rtabsrc))) ->
+            let -- create current function to be included in a RCombinedOp operation 
+                currfunc :: UnaryRTableOperation
+                currfunc = insertRTabToRTab rtabsrc -- this returns an RTable -> RTable function
+                -- get previous RCombinedOp operation and table expressions
+                (prevOperation, prevTXEleft, prevTXEright) = evalROpExpr restExpression
+
+            -- the current RCombinedOp is produced by composing the current function with the previous function
+            in case prevOperation of
+                    -- in this case the previous operation is a valid non-empty operation and must be composed with the current one
+                    RCombinedOp {rcombOp = prevfunc} -> (RCombinedOp {rcombOp = currfunc . prevfunc}, prevTXEleft, EmptyTab)
+                    -- in this case the current Operation is the last one (the previous is just empty and must be ignored)
+                    ROperationEmpty ->  (RCombinedOp {rcombOp = currfunc}, TXE tabExpr, EmptyTab)                    
+
+        -- this is the Upsert (Update+Insert, aka Merge) Operation.
+        Upsert (    
+                    MergeInto tabExpr ( 
+                                        Using (TabSrc srcTab) ( 
+                                                                MergeOn upsPred ( 
+                                                                                   WhenMatchedThen ( 
+                                                                                                        UpdateCols cols (FilterBy fpred)
+                                                                                                    )
+                                                                                )
+                                                              )
+                                      )
+                ) ->
+            let -- create current function to be included in a RCombinedOp operation 
+                currfunc :: UnaryRTableOperation
+                currfunc = upsertRTab srcTab upsPred cols fpred -- this returns an RTable -> RTable function
+                -- get previous RCombinedOp operation and table expressions
+                (prevOperation, prevTXEleft, prevTXEright) = evalROpExpr restExpression
+
+            -- the current RCombinedOp is produced by composing the current function with the previous function
+            in case prevOperation of
+                    -- in this case the previous operation is a valid non-empty operation and must be composed with the current one
+                    RCombinedOp {rcombOp = prevfunc} -> (RCombinedOp {rcombOp = currfunc . prevfunc}, prevTXEleft, EmptyTab)
+                    -- in this case the current Operation is the last one (the previous is just empty and must be ignored)
+                    ROperationEmpty ->  (RCombinedOp {rcombOp = currfunc}, TXE tabExpr, EmptyTab)                    
+
+
 -- | turns the  list of agg operation expressions to a list of RAggOperation data type
 aggOpExprToAggOp :: [AggOp] -> [RAggOperation]
 aggOpExprToAggOp [] = []
 aggOpExprToAggOp (aggopExpr : rest) = 
     let aggop = case aggopExpr of
-                    Sum srcCol (As trgCol)      -> (raggSum srcCol trgCol)
-                    Count srcCol (As trgCol)    -> (raggCount srcCol trgCol)
-                    Min srcCol (As trgCol)      -> (raggMin srcCol trgCol)
-                    Max srcCol (As trgCol)      -> (raggMax srcCol trgCol)
-                    Avg srcCol (As trgCol)      -> (raggAvg srcCol trgCol)
+                    Sum srcCol (As trgCol)                  -> (raggSum srcCol trgCol)
+                    Count srcCol (As trgCol)                -> (raggCount srcCol trgCol)
+                    CountDist srcCol (As trgCol)            -> (raggCountDist srcCol trgCol)
+                    CountStar (As trgCol)                   -> (raggCountStar trgCol)                                        
+                    Min srcCol (As trgCol)                  -> (raggMin srcCol trgCol)
+                    Max srcCol (As trgCol)                  -> (raggMax srcCol trgCol)
+                    Avg srcCol (As trgCol)                  -> (raggAvg srcCol trgCol)
+                    StrAgg srcCol (As trgCol) delimiter     -> (raggStrAgg srcCol trgCol delimiter)
                     GenAgg srcCol (As trgCol) (AggBy aggf)  -> (raggGenericAgg aggf srcCol trgCol)
     in aggop : (aggOpExprToAggOp rest)
 
-            
  
 -- and then run the produced ETLMapping
 finalRTable = etl $ evalJulius myEtlExpr
@@ -1349,12 +1663,11 @@ --  GenUnaryOp (On Tab rtab1) $ ByUnaryOp (addSurrogateKeyJ "TxSK" 0)
 -- @
 -- 
-addSurrogateKeyJ :: Integral a =>    
-       ColumnName    -- ^ The name of the surrogate key column
-    -- -> Integer       -- ^ The initial value of the Surrogate Key will be the value of this parameter    
-    -> a             -- ^ The initial value of the Surrogate Key will be the value of this parameter    
-    -> RTable        -- ^ Input RTable
-    -> RTable        -- ^ Output RTable
+addSurrogateKeyJ :: Integral a 
+                 =>  ColumnName    -- ^ The name of the surrogate key column
+                 -> a             -- ^ The initial value of the Surrogate Key will be the value of this parameter    
+                 -> RTable        -- ^ Input RTable
+                 -> RTable        -- ^ Output RTable
 addSurrogateKeyJ cname initVal  =   
     updateSKvalue . (addColumn cname (RInt (fromIntegral initVal))) 
             where

@@ -282,12 +282,14 @@     ,RTimestamp (..)
     -- ** RTable Metadata Data Types
     ,RTableMData (..)
-    ,RTupleMData (..)
-    ,ColumnInfo (..)    
+    ,RTupleMData (..)    
+    ,ColumnInfo (..) 
+    ,ColumnOrder   
     ,Name
     ,ColumnName
     ,RTableName
     ,ColumnDType (..)
+    ,Delimiter
 
     -- * Type Classes for "Tabular Data"
     ,RTabular (..)
@@ -303,14 +305,18 @@     ,raggGenericAgg
     ,raggSum
     ,raggCount
+    ,raggCountDist
+    ,raggCountStar
     ,raggAvg
     ,raggMax
-    ,raggMin    
+    ,raggMin
+    ,raggStrAgg    
 
     -- ** Predicates
     ,RPredicate
     ,RGroupPredicate
     ,RJoinPredicate
+    ,RUpsertPredicate (..)
 
     -- ** Operation Execution
     ,runUnaryROperation
@@ -398,8 +404,13 @@     ,rJ
     ,runFullOuterJoin
     ,foJ
+    ,sJ
+    ,runSemiJoin
+    ,aJ
+    ,runAntiJoin
     ,joinRTuples    
     ,runUnion
+    ,runUnionAll
     ,u
     ,runIntersect
     ,i
@@ -424,7 +435,9 @@     ,decodeColValue
     -- ** Date/Time
     ,toRTimestamp
-    ,createRTimestamp    
+    ,createRTimestamp
+    ,toUTCTime
+    ,fromUTCTime    
     ,rTimestampToRText
     ,stdTimestampFormat
     ,stdDateFormat
@@ -464,15 +477,21 @@     ,toText
     ,fromText    
     -- ** Container Functions
-    ,rtabMap
+    ,rtabMap    
     ,rtabFoldr'
     ,rtabFoldl'
+    ,rtupleMap
+    ,rtupleMapWithKey
     ,rdatatypeFoldr'
     ,rdatatypeFoldl'
     -- * Modify RTable (DML)
     ,insertAppendRTab    
     ,insertPrependRTab
-    ,updateRTab  
+    ,insertRTabToRTab
+    ,deleteRTab    
+    ,updateRTab    
+    ,upsertRTab  
+    ,updateRTuple
     ,upsertRTuple    
     -- * Create/Alter RTable (DDL)
     ,emptyRTable
@@ -482,21 +501,29 @@     ,removeColumn    
     ,emptyRTuple
     ,createNullRTuple
-    ,createRtuple
+    ,createRTuple
     ,rtupleFromList    
     ,createRDataType
     -- * Metadata Functions
     ,createRTableMData    
-    ,getColumnNamesfromRTab
-    ,getColumnNamesfromRTuple
+    ,getColumnNamesFromRTab
+    ,getColumnNamesFromRTuple
+    ,getColumnInfoFromRTab
+    ,getColumnInfoFromRTuple
+    ,getTheType
     ,listOfColInfoRDataType
     ,toListColumnName
     ,toListColumnInfo
+    ,rtabsSameStructure
+    ,rtuplesSameStructure
+    ,getUniqueColumnNamesAfterJoin
     -- * Exceptions
     ,ColumnDoesNotExist (..)
+    ,ConflictingRTableStructures (..)
+    ,EmptyInputStringsInToRTimestamp (..)    
     ,UnsupportedTimeStampFormat (..)
+    ,UniquenessViolationInUpsert (..)
     --,RTimestampFormatLengthMismatch (..)
-    ,EmptyInputStringsInToRTimestamp (..)
 
     -- * RTable IO Operations
     -- ** RTable Printing and Formatting
@@ -592,7 +619,7 @@ import qualified Data.Dynamic as D  -- https://hackage.haskell.org/package/base-4.9.1.0/docs/Data-Dynamic.html
 
 -- Data.List
-import Data.List (last, all, elem, break, span, map, null, zip, zipWith, elemIndex, sortOn, union, intersect, (\\), take, length, repeat, groupBy, sort, sortBy, foldl', foldr, foldr1, foldl',head, findIndex, tails, isPrefixOf)
+import Data.List (find, filter, last, all, elem, break, span, map, null, zip, zipWith, elemIndex, sortOn, union, intersect, (\\), take, length, repeat, groupBy, sort, sortBy, foldl', foldr, foldr1, foldl',head, findIndex, tails, isPrefixOf)
 -- Data.Maybe
 import Data.Maybe (fromJust, fromMaybe)
 -- Data.Char
@@ -612,6 +639,9 @@ import Control.DeepSeq
 import Data.String.Utils (replace)
 
+import Data.Time.Clock (UTCTime(..), diffTimeToPicoseconds, secondsToDiffTime)
+import Data.Time.Calendar (Day, toGregorian, fromGregorian)
+
 -- import Control.Monad.IO.Class (liftIO)
 
 
@@ -716,7 +746,7 @@ 
 -- | This is used only for metadata purposes (see 'ColumnInfo'). The actual data type of a value is an RDataType
 -- The Text component of Date and Timestamp data constructors is the date format e.g., "DD\/MM\/YYYY", "DD\/MM\/YYYY HH24:MI:SS"
-data ColumnDType = Integer | Varchar | Date Text | Timestamp Text | Double deriving (Show, Eq)
+data ColumnDType = UknownType | Integer | Varchar | Date Text | Timestamp Text | Double  deriving (Show, Eq)
 
 -- | Definition of the Relational Data Type. This is the data type of the values stored in each 'RTable'.
 -- This is a strict data type, meaning whenever we evaluate a value of type 'RDataType', 
@@ -762,7 +792,7 @@     -- RInt i == _ = False
     RText t1 == RText t2 = t1 == t2
     -- RText t1 == _ = False
-    RDate t1 s1 == RDate t2 s2 = (t1 == t1) && (s1 == s2)
+    RDate t1 s1 == RDate t2 s2 = toRTimestamp (unpack s1) (unpack t1) == toRTimestamp (unpack s2) (unpack t2)   -- (t1 == t1) && (s1 == s2)
     -- RDate t1 s1 == _ = False
     RTime t1 == RTime t2 = t1 == t2
     -- RTime t1 == _ = False
@@ -781,18 +811,18 @@     x /= y = not (x == y) 
 
 -- Need to explicitly specify due to "Null logic" (see Eq)
-instance Ord RDataType where
-    Null <= _ = False
-    _ <= Null = False
+instance Ord RDataType where    
+    compare Null _ = GT     --    Null <= _ = False
+    compare _ Null = GT     --  _ <= Null = False
     -- Null <= Null = False -- Comment out due to redundant warning
-    RInt i1 <= RInt i2 = i1 <= i2
-    RText t1 <= RText t2 = t1 <= t2
-    RDate t1 s1 <= RDate t2 s2 = (t1 <= t1) && (s1 == s2)
-    RTime t1 <= RTime t2 = t1 <= t2
-    RTime t1 <= _ = False
-    RDouble d1 <= RDouble d2 = d1 <= d2
+    compare (RInt i1) (RInt i2) = compare i1 i2                                             --  RInt i1 <= RInt i2 = i1 <= i2
+    compare (RText t1) (RText t2) = compare t1 t2                                           --  RText t1 <= RText t2 = t1 <= t2
+    compare (RDate t1 s1) (RDate t2 s2) = compare (toRTimestamp (unpack s1) (unpack t1)) (toRTimestamp (unpack s2) (unpack t2)) --  RDate t1 s1 <= RDate t2 s2 = (t1 <= t1) && (s1 == s2)
+    compare (RTime t1) (RTime t2) = compare t1 t2                                           --  RTime t1 <= RTime t2 = t1 <= t2
+    -- RTime t1 <= _ = False
+    compare (RDouble d1) (RDouble d2) = compare d1 d2                                       --  RDouble d1 <= RDouble d2 = d1 <= d2
     -- anything else is just False
-    _ <= _ = False
+    compare _ _ = GT                                                                        --  _ <= _ = False
 
 
 -- | Use this function to compare an RDataType with the Null value because due to Null logic
@@ -864,19 +894,39 @@ stdDateFormat = "DD/MM/YYYY"
 
 -- | Get the Column Names of an RTable
-getColumnNamesfromRTab :: RTable -> [ColumnName]
-getColumnNamesfromRTab rtab = getColumnNamesfromRTuple $ headRTup rtab
+getColumnNamesFromRTab :: RTable -> [ColumnName]
+getColumnNamesFromRTab rtab = getColumnNamesFromRTuple $ headRTup rtab
 
+-- | Returns the Column Names of an RTuple
+getColumnNamesFromRTuple :: RTuple -> [ColumnName]
+getColumnNamesFromRTuple t = HM.keys t
+
+-- Get the column metadata of an 'RTable'
+getColumnInfoFromRTab :: RTable -> [ColumnInfo]
+getColumnInfoFromRTab rtab = getColumnInfoFromRTuple $ headRTup rtab
+
+-- Get the column metadata of an 'RTuple'
+getColumnInfoFromRTuple :: RTuple -> [ColumnInfo]
+getColumnInfoFromRTuple  t = 
+   HM.elems $ HM.mapWithKey (\c v -> ColumnInfo { name = c, dtype = getTheType v}) t 
+
+-- | Take a column value and return its type
+getTheType :: RDataType -> ColumnDType
+getTheType v = 
+    case v of
+        RInt _                           ->  Integer
+        RText _                          ->  Varchar
+        RDate {rdate = d, dtformat = f}  ->  Date f 
+        RTime _                          ->  Timestamp (pack stdTimestampFormat)
+        RDouble _                        ->  Double
+        Null                             ->  UknownType
+
 -- | Get the first RTuple from an RTable
 headRTup ::
         RTable
     ->  RTuple
 headRTup = V.head    
 
--- | Returns the Column Names of an RTuple
-getColumnNamesfromRTuple :: RTuple -> [ColumnName]
-getColumnNamesfromRTuple t = HM.keys t
-
 -- | Returns the value of an RTuple column based on the ColumnName key
 --   if the column name is not found, then it returns Nothing
 rtupLookup ::
@@ -1036,16 +1086,6 @@         else
             V.map (\t -> upsertRTuple cName (decodeColValue cName searchVal returnVal defaultVal ignoreInd t) t) tab   
 
--- | Upsert (update/insert) an RTuple at a specific column specified by name with a value
--- If the cname key is not found then the (columnName, value) pair is inserted. If it exists
--- then the value is updated with the input value.
-upsertRTuple ::
-           ColumnName  -- ^ key where the upset will take place
-        -> RDataType   -- ^ new value
-        -> RTuple      -- ^ input RTuple
-        -> RTuple      -- ^ output RTuple
-upsertRTuple cname newVal tupsrc = HM.insert cname newVal tupsrc
-
 -- newtype NumericRDT = NumericRDT { getRDataType :: RDataType } deriving (Eq, Ord, Read, Show, Num)
 
 
@@ -1343,7 +1383,25 @@                     then throw $ UnsupportedTimeStampFormat fmt
                     else parseFormat2 restFormatFinal restTstampFinal newMap
 
+-- | Convert an 'RTimestamp' value to a Universal Time value ('UTCTime')
+toUTCTime :: RTimestamp -> UTCTime
+toUTCTime (RTimestampVal yy mm dd hh24 mins ss) =
+    let day = fromGregorian (toInteger $ yy::Integer) mm dd
+        secs = secondsToDiffTime $ (toInteger $ hh24*60*60 + mm*60 + ss  :: Integer)
+    in UTCTime day secs 
 
+-- | Convert a Universal Time value ('UTCTime') to an 'RTimestamp' value
+fromUTCTime :: UTCTime -> RTimestamp
+fromUTCTime utc = 
+    let UTCTime day s = utc
+        secs =  round $ ((1.0e-12::Double) *) $ (fromInteger $ diffTimeToPicoseconds s)  ::Int
+        (yy, mm, dd) = toGregorian day
+        hh24 = (secs `div` 60) `div` 60
+        mins =  (secs `div` 60) `rem` 60
+        ss = (secs `rem` (60*60)) `rem` 60
+    in -- RTimestampVal (fromIntegral yy) mm dd hh24 mins ss
+        RTimestampVal {year = (fromIntegral yy), month = mm, day = dd, hours24 = hh24, minutes = mins, seconds = ss}
+
 -- | Search for the first occurence of a substring within a 'String' and return the 1st character position,
 -- or 'Nothing' if the substring is not found.
 ---- See :  
@@ -1504,7 +1562,7 @@ isText _ = False
 
 -- | Standard timestamp format. For example: \"DD/MM/YYYY HH24:MI:SS\"
-stdTimestampFormat = "DD/MM/YYYY HH24:MI:SS"
+stdTimestampFormat = "DD/MM/YYYY HH24:MI:SS" :: String
 
 -- | rTimeStampToText: converts an RTimestamp value to RText
 -- Valid input formats are:
@@ -1662,12 +1720,30 @@                                           --   Since an RTuple is implemented as a HashMap ColumnName RDataType, ordering of columns has no meaning.
                                           --   However, with this columns we can "pretend" that there is a fixed column order in each RTuple.
                         ,dtype :: ColumnDType
-                    } deriving (Show, Eq)
+                    } deriving (Show,Eq)
 
+-- | Define equality for two 'ColumnInfo' structures
+-- For two column two have \"equal structure\" they must have the same name
+-- and the same type. If one of the two (or both) have an 'UknownType', then they are still considered of equal structure.
+{-
+instance Eq (ColumnInfo) where
+    (==) ci1 ci2 = 
+        if (name ci1) == (name ci2)
+            then
+                if ((dtype ci1) == (dtype ci2))
+                    ||
+                    (dtype ci1 == UknownType)
+                    ||
+                    (dtype ci2 == UknownType)
+                    then True
+                    else False
+            else
+                False
+-}
 
--- | Creates a list of the form [(ColumnInfo, RDataType)]  from a list of ColumnInfo and an RTuple. The returned list respects the order of the [ColumnInfo]
-    -- Prelude.zip listOfColInfo (Prelude.map (snd) $ HM.toList rtup)  -- this code does NOT guarantee that HM.toList will return the same column order as [ColumnInfo]
-listOfColInfoRDataType :: [ColumnInfo] -> RTuple -> [(ColumnInfo, RDataType)]  -- this code does guarantees that RDataTypes will be in the same column order as [ColumnInfo], i.e., the correct RDataType for the correct column
+-- | Creates a list of the form [(ColumnInfo, RDataType)]  from a list of ColumnInfo and an RTuple. The returned list respects the order of the [ColumnInfo].
+-- It guarantees that RDataTypes will be in the same column order as [ColumnInfo], i.e., the correct RDataType for the correct column
+listOfColInfoRDataType :: [ColumnInfo] -> RTuple -> [(ColumnInfo, RDataType)]  
 listOfColInfoRDataType (ci:[]) rtup = [(ci, rtup HM.!(name ci))]  -- rt HM.!(name ci) -> this returns the RDataType by column name
 listOfColInfoRDataType (ci:colInfos) rtup = (ci, rtup HM.!(name ci)):listOfColInfoRDataType colInfos rtup
 
@@ -1811,7 +1887,9 @@                                               -- @
                                               -- is a valid join predicate. I.e., a function which returns 'True' when two 'RTuples' must be paired)
     | RLeftJoin { jpred :: RJoinPredicate }   -- ^ Left Outer Join    
-    | RRightJoin { jpred :: RJoinPredicate }  -- ^ Right Outer Join         
+    | RRightJoin { jpred :: RJoinPredicate }  -- ^ Right Outer Join
+    | RSemiJoin { jpred :: RJoinPredicate }     -- ^ Semi-Join
+    | RAntiJoin { jpred :: RJoinPredicate }     -- ^ Anti-Join    
     | RAggregate { aggList :: [RAggOperation] -- ^ list of aggregates 
                  } -- ^ Performs aggregation operations on specific columns and returns a singleton RTable
     | RGroupBy  { 
@@ -1861,6 +1939,18 @@ -- | The Join Predicate. It defines when two 'RTuple's should be paired.
 type RJoinPredicate = RTuple -> RTuple -> Bool
 
+-- | The Upsert Predicate. It defines when two 'RTuple's should be paired in a merge operation.
+-- The matching predicate must be applied on a specific set of matching columns. The source 'RTable'
+-- in the Upsert operation must return a unique set of 'RTuple's, if grouped by this set of matching columns.
+-- Otherwise an exception ('UniquenessViolationInUpsert') is thrown.
+data RUpsertPredicate = RUpsertPredicate {
+                            matchCols :: [ColumnName]
+                            ,matchPred :: RTuple -> RTuple -> Bool
+                        }
+
+-- type RUpsertPredicate = RTuple -> RTuple -> Bool
+                        
+
 -- | The Group By Predicate
 -- It defines the condition for two 'RTuple's to be included in the same group.
 type RGroupPredicate = RTuple -> RTuple -> Bool
@@ -1903,9 +1993,53 @@ raggGenericAgg aggf src trg = RAggOperation {
                  sourceCol = src
                 ,targetCol = trg                                 
-                ,aggFunc = \rtab -> createRtuple [(trg, aggf src rtab)]  
+                ,aggFunc = \rtab -> createRTuple [(trg, aggf src rtab)]  
         }
 
+type Delimiter = String
+
+-- | The StrAgg aggregate operation
+-- This is known as \"string_agg\"" in Postgresql and \"listagg\" in Oracle.
+-- It aggregates the values of a text 'RDataType' column with a specified delimiter
+raggStrAgg ::
+        ColumnName -- ^ source column
+    ->  ColumnName -- ^ target column
+    ->  Delimiter  -- ^ delimiter string 
+    ->  RAggOperation
+raggStrAgg src trg delimiter =  RAggOperation {
+                 sourceCol = src
+                ,targetCol = trg                       
+                ,aggFunc = \rtab -> createRTuple [(trg, strAggFold delimiter src rtab)]  
+        }
+
+-- | A helper function that implements the basic fold for the raggStrAgg aggregation        
+strAggFold :: Delimiter -> AggFunction
+strAggFold dlmt col rtab = 
+    rdatatypeFoldr' ( \rtup accValue -> 
+        if isNotNull (rtup <!> col)  && (isNotNull accValue)
+            then
+                rtup <!> col `rdtappend` (RText delimiter) `rdtappend` accValue
+            else
+                --if (getRTupColValue src) rtup == Null && accValue /= Null 
+                if isNull (rtup <!> col) && (isNotNull accValue)
+                    then
+                        accValue  -- ignore Null value
+                    else
+                        --if (getRTupColValue src) rtup /= Null && accValue == Null 
+                        if isNotNull (rtup <!> col) && (isNull accValue)
+                            then
+                                case isText (rtup <!> col) of
+                                    True ->  (rtup <!> col) 
+                                    False -> Null
+                            else
+                                Null -- agg of Nulls is Null
+                    )
+                    Null
+                    rtab
+    where
+        delimiter = pack dlmt -- convert String to Text
+
+
 -- | The Sum aggregate operation
 raggSum :: 
         ColumnName -- ^ source column
@@ -1914,11 +2048,11 @@ raggSum src trg = RAggOperation {
                  sourceCol = src
                 ,targetCol = trg                                 
-                ,aggFunc = \rtab -> createRtuple [(trg, sumFold src rtab)]  
+                ,aggFunc = \rtab -> createRTuple [(trg, sumFold src rtab)]  
         }
 
 -- | A helper function in raggSum that implements the basic fold for sum aggregation        
-sumFold :: AggFunction --ColumnName -> ColumnName -> RTable -> RDataType
+sumFold :: AggFunction -- ColumnName -> RTable -> RDataType
 sumFold src rtab =         
     V.foldr' ( \rtup accValue ->                                                                     
                     --if (getRTupColValue src) rtup /= Null && accValue /= Null
@@ -1943,6 +2077,7 @@ 
 
 -- | The Count aggregate operation
+-- Count aggregation (no distinct)
 raggCount :: 
         ColumnName -- ^ source column
     ->  ColumnName -- ^ target column
@@ -1950,11 +2085,12 @@ raggCount src trg = RAggOperation {
                  sourceCol = src
                 ,targetCol = trg
-                ,aggFunc = \rtab -> createRtuple [(trg, countFold src rtab)]  
+                ,aggFunc = \rtab -> createRTuple [(trg, countFold src rtab)]  
         }
 
+
 -- | A helper function in raggCount that implements the basic fold for Count aggregation        
-countFold :: AggFunction -- ColumnName -> ColumnName -> RTable -> RDataType
+countFold :: AggFunction -- ColumnName -> RTable -> RDataType
 countFold src rtab =         
     V.foldr' ( \rtup accValue ->                                                                     
                             --if (getRTupColValue src) rtup /= Null && accValue /= Null
@@ -1976,6 +2112,63 @@              ) (Null) rtab
 
 
+-- | The CountStar aggregate operation 
+--  Returns the number of 'RTuple's in the 'RTable' (i.e., @count(*)@ in SQL) 
+raggCountStar ::         
+        ColumnName -- ^ target column to save the result aggregated value
+    ->  RAggOperation
+raggCountStar trg  = RAggOperation {
+                sourceCol = ""  -- no source column required
+                ,targetCol = trg
+                ,aggFunc = \rtab -> createRTuple [(trg, countStarFold rtab)]  
+        }
+-- | A helper function in raggCountStar that implements the basic fold for CountStar aggregation        
+countStarFold :: RTable -> RDataType
+countStarFold rtab = RInt $ toInteger $ Data.List.length $ rtableToList rtab
+
+
+-- | The CountDist aggregate operation 
+-- Count distinct aggregation (i.e., @count(distinct col)@ in SQL). Returns the distinct number of values for this column.
+raggCountDist :: 
+        ColumnName -- ^ source column
+    ->  ColumnName -- ^ target column
+    ->  RAggOperation
+raggCountDist src trg = RAggOperation {
+                 sourceCol = src
+                ,targetCol = trg
+                ,aggFunc = \rtab -> createRTuple [(trg, countDistFold src rtab)]  
+        }
+
+-- | A helper function in raggCountDist that implements the basic fold for CountDist aggregation        
+countDistFold :: AggFunction -- ColumnName -> ColumnName -> RTable -> RDataType
+countDistFold src rtab = 
+    let
+        -- change the input rtable to fold, to be the distinct list of values
+        -- implement it with a group by on the src column
+        rtabDist = runGroupBy   (\t1 t2 -> t1 <!> src == t2 <!> src)
+                                [raggCount src "dummy"]  -- this will be omitted anyway
+                                [src]
+                                rtab
+    in V.foldr' ( \rtup accValue ->                                                                     
+                            --if (getRTupColValue src) rtup /= Null && accValue /= Null
+                            if (isNotNull $ (getRTupColValue src) rtup)  && (isNotNull accValue)  
+                                then
+                                    RInt 1 + accValue
+                                else
+                                    --if (getRTupColValue src) rtup == Null && accValue /= Null 
+                                    if (isNull $ (getRTupColValue src) rtup) && (isNotNull accValue)  
+                                        then
+                                            accValue  -- ignore Null value
+                                        else
+                                            --if (getRTupColValue src) rtup /= Null && accValue == Null 
+                                            if (isNotNull $ (getRTupColValue src) rtup) && (isNull accValue)
+                                                then
+                                                    RInt 1  -- ignore so far Null agg result
+                                                else
+                                                    Null -- agg of Nulls is Null
+             ) (Null) rtabDist
+
+
 -- | The Average aggregate operation
 raggAvg :: 
         ColumnName -- ^ source column
@@ -1984,7 +2177,7 @@ raggAvg src trg = RAggOperation {
                  sourceCol = src
                 ,targetCol = trg
-                ,aggFunc = \rtab -> createRtuple [(trg, let 
+                ,aggFunc = \rtab -> createRTuple [(trg, let 
                                                              sum = sumFold src rtab
                                                              cnt =  countFold src rtab
                                                         in case (sum,cnt) of
@@ -2002,7 +2195,7 @@ raggMax src trg = RAggOperation {
                  sourceCol = src
                 ,targetCol = trg
-                ,aggFunc = \rtab -> createRtuple [(trg, maxFold src rtab)]  
+                ,aggFunc = \rtab -> createRTuple [(trg, maxFold src rtab)]  
         }        
 
 
@@ -2037,7 +2230,7 @@ raggMin src trg = RAggOperation {
                  sourceCol = src
                 ,targetCol = trg
-                ,aggFunc = \rtab -> createRtuple [(trg, minFold src rtab)]  
+                ,aggFunc = \rtab -> createRTuple [(trg, minFold src rtab)]  
         }        
 
 -- | A helper function in raggMin that implements the basic fold for Min aggregation        
@@ -2116,6 +2309,8 @@         RInJoin    { jpred = jpredicate } -> runInnerJoinO jpredicate irtab1 irtab2
         RLeftJoin  { jpred = jpredicate } -> runLeftJoin jpredicate irtab1 irtab2
         RRightJoin { jpred = jpredicate } -> runRightJoin jpredicate irtab1 irtab2
+        RSemiJoin    { jpred = jpredicate } -> runSemiJoin jpredicate irtab1 irtab2
+        RAntiJoin    { jpred = jpredicate } -> runAntiJoin jpredicate irtab1 irtab2        
         RUnion -> runUnion irtab1 irtab2
         RInter -> runIntersect irtab1 irtab2
         RDiff  -> runDiff irtab1 irtab2  
@@ -2162,10 +2357,10 @@ createSingletonRTable rt = V.singleton rt
 
 -- | createRTuple: Create an Rtuple from a list of column names and values
-createRtuple ::
+createRTuple ::
       [(ColumnName, RDataType)]  -- ^ input list of (columnname,value) pairs
     -> RTuple 
-createRtuple l = HM.fromList l
+createRTuple l = HM.fromList l
 
 
 
@@ -2177,7 +2372,9 @@ createNullRTuple cnames = HM.fromList $ zipped
     where zipped = Data.List.zip cnames (Data.List.take (Data.List.length cnames) (repeat Null))
 
--- | Returns True if the input RTuple is a Null RTuple, otherwise it returns False
+-- | Returns 'True' if the input 'RTuple' is a Null RTuple, otherwise it returns 'False'
+-- Note that a Null RTuple has all its values equal with 'Null' but it still has columns. This is different from an empty 'RTuple', which
+-- is an 'RTuple' withi no columns and no values whatsoever. See 'isRTupEmpty'.
 isNullRTuple ::
        RTuple 
     -> Bool    
@@ -2229,10 +2426,18 @@ rdatatypeFoldl' f accum rtab = V.foldl' f accum rtab
 
 
--- | Map function over an RTable
+-- | Map function over an 'RTable'.
 rtabMap :: (RTuple -> RTuple) -> RTable -> RTable
 rtabMap f rtab = V.map f rtab 
 
+-- | O(n) Transform this 'RTuple' by applying a function to every value
+rtupleMap :: (RDataType -> RDataType) -> RTuple -> RTuple
+rtupleMap f t = HM.map f t 
+
+-- | O(n) Transform this 'RTuple' by applying a function to every value
+rtupleMapWithKey :: (ColumnName -> RDataType -> RDataType) -> RTuple -> RTuple
+rtupleMapWithKey f t = HM.mapWithKey f t
+
 -- * ########## RTable Relational Operations ##############
 
 -- | Number of RTuples returned by an RTable operation
@@ -2381,7 +2586,67 @@ restrictNrows n r1 = V.take n r1
 -}
 
--- | RTable Inner Join Operator. A short name for the 'runInnerJoinO' function
+-- | 'RTable' anti-join operator. A short name for the 'runAntiJoin' function
+aJ = runAntiJoin
+
+-- | Implements the anti-Join operation between two RTables (any type of join predicate is allowed)
+-- It returns the 'RTuple's from the left 'RTable' that DONT match with the right 'RTable'.
+runAntiJoin ::
+    RJoinPredicate
+    -> RTable
+    -> RTable
+    -> RTable
+runAntiJoin jpred tabDriver tabProbed = 
+    if isRTabEmpty tabDriver || isRTabEmpty tabProbed
+        then
+            emptyRTable
+        else
+            d tabDriver $ sJ jpred tabDriver tabProbed 
+{-            do 
+                rtupDrv <- tabDriver
+                -- this is the equivalent of a nested loop with tabDriver playing the role of the driving table and tabProbed the probed table
+                V.foldr' (\t accum -> 
+                            if (not $ jpred rtupDrv t) 
+                                then 
+                                    -- insert joined tuple to result table (i.e. the accumulator)
+                                    insertAppendRTab (rtupDrv) accum
+                                else 
+                                    -- keep the accumulator unchanged
+                                    accum
+                        ) emptyRTable tabProbed 
+-}
+
+-- | 'RTable' semi-join operator. A short name for the 'runSemiJoin' function
+sJ = runSemiJoin
+
+-- | Implements the semi-Join operation between two RTables (any type of join predicate is allowed)
+-- It returns the 'RTuple's from the left 'RTable' that match with the right 'RTable'.
+-- Note that if an 'RTuple' from the left 'RTable' matches more than one 'RTuple's from the right 'RTable'
+-- the semi join operation will return only a single 'RTuple'.    
+runSemiJoin ::
+    RJoinPredicate
+    -> RTable
+    -> RTable
+    -> RTable
+runSemiJoin jpred tabDriver tabProbed = 
+    if isRTabEmpty tabDriver || isRTabEmpty tabProbed
+        then
+            emptyRTable
+        else 
+            do 
+                rtupDrv <- tabDriver
+                -- this is the equivalent of a nested loop with tabDriver playing the role of the driving table and tabProbed the probed table
+                V.foldr' (\t accum -> 
+                            if (jpred rtupDrv t) 
+                                then 
+                                    -- insert joined tuple to result table (i.e. the accumulator)
+                                    insertAppendRTab (rtupDrv) accum
+                                else 
+                                    -- keep the accumulator unchanged
+                                    accum
+                        ) emptyRTable tabProbed 
+
+-- | 'RTable' Inner Join Operator. A short name for the 'runInnerJoinO' function
 iJ = runInnerJoinO
 
 -- | Implements an Inner Join operation between two RTables (any type of join predicate is allowed)
@@ -2551,19 +2816,63 @@                     -- we know that both the preservingTab and tab are non empty 
                     let 
                         unionFstPart = iJ jpred preservingTab tab 
+
+                        -- debug
+                        -- !dummy1 = trace ("unionFstPart:\n" ++ show unionFstPart) True
+
                         -- project only the preserving tab's columns
-                        fstPartProj = p (getColumnNamesfromRTab preservingTab) unionFstPart
+                        fstPartProj = p (getColumnNamesFromRTab preservingTab) unionFstPart
 
                         -- the second part are the rows from the preserving table that dont join
                         -- we will use the Difference operations for this
                         unionSndPart = 
                             let 
                                 difftab = d preservingTab fstPartProj -- unionFstPart
+
                                 -- now enhance the result with the columns of the right table
-                            in iJ (\t1 t2 -> True) difftab (createSingletonRTable $ createNullRTuple $ (getColumnNamesfromRTab tab))
+                            in iJ (\t1 t2 -> True) difftab (createSingletonRTable $ createNullRTuple $ (getColumnNamesFromRTab tab))
+                        -- debug
+                        -- !dummy2 = trace ("unionSndPart :\n" ++ show unionSndPart) True
+                        -- !dummy3 = trace ("u unionFstPart unionSndPart :\n" ++ (show $ u unionFstPart unionSndPart)) True
+
+                            {-   -- now enhance the result with the columns of the right table
+                                joinedColumnsTab = iJ (\t1 t2 -> True) difftab (createSingletonRTable $ createNullRTuple $ (getColumnNamesFromRTab tab))
+                                -- get only the columns from the two rtables that dont overlap
+                                finalListOfColumns = getUniqueColumnNames (getColumnNamesFromRTab preservingTab) (getColumnNamesFromRTab tab) 
+                                -- project only the columns from both rtables that dont overlap
+                                -- otherwise, the union will hit an "ConflictingRTableStructures "Cannot run: Union, due to conflicting RTable structures" exception.
+                            in p finalListOfColumns joinedColumnsTab
+                            -}
                     in u unionFstPart unionSndPart
 
 
+-- | Receives two lists of 'ColumnName's and returns the unique list of 'ColumnName's 
+-- after concatenating the two and removing the names from the second one that are a prefix of the first one.
+-- This function is intended to dedublicate common columns after a join (see 'ij'), where "ColA" for example,
+-- will also appear as "ColA_1". This function DOES NOT dedublicate columns "ColA" and "ColAsomeSuffix", only
+-- cases like this one "ColName_Num" (e.g., ColName_1, ColName_2, etc.)
+-- Here is an example:
+--
+-- >>> getUniqueColumnNames ["ColA","ColB"] ["ColC","ColA", "ColA_1", "ColA_2", "ColA_A", "ColA_hello", "ColAhello"]
+-- >>> ["ColA","ColB","ColC","ColA_A","ColA_hello","ColAhello"]
+--
+getUniqueColumnNamesAfterJoin :: [ColumnName] -> [ColumnName] -> [ColumnName]
+getUniqueColumnNamesAfterJoin cl1 cl2 = 
+    -- cl1  ++ Data.List.filter (\n2 -> and $ Data.List.map (\n1 -> not $ T.isPrefixOf n1 n2) cl1) cl2
+    cl1  ++ Data.List.filter (\n2 -> and $ Data.List.map (\n1 -> not $ isMyPrefixOf n1 n2) cl1) cl2
+    where
+        -- We want a prefix test that will return:
+        -- isMyPrefixOf "ColA" "ColA_1" == True
+        -- isMyPrefixOf "ColA" "ColA_lala" == False
+        -- isMyPrefixOf "ColA" "ColAlala" == False
+        isMyPrefixOf :: ColumnName -> ColumnName -> Bool
+        isMyPrefixOf cn1 cn2 = 
+            let
+                -- rip off suffixes of the form "_Num" e.g., "Col_1", "Col_2"  -> "Col", "Col"
+                cn2_new = T.dropWhileEnd (\c -> c == '_') $ T.dropWhileEnd (\c -> isDigit c) cn2
+                -- and now compare for equality
+            in cn1 == cn2_new
+
 -- | RTable Right Outer Join Operator. A short name for the 'runRightJoin' function
 rJ = runRightJoin
 
@@ -2604,7 +2913,7 @@ 
                         -- project only the preserving tab's columns
                         fstPartProj = 
-                            p (getColumnNamesfromRTab preservingTab) unionFstPart
+                            p (getColumnNamesFromRTab preservingTab) unionFstPart
 
                         -- the second part are the rows from the preserving table that dont join
                         -- we will use the Difference operations for this
@@ -2613,7 +2922,7 @@                                 difftab = 
                                     d preservingTab   fstPartProj -- unionFstPart 
                                 -- now enhance the result with the columns of the left table
-                            in iJ (\t1 t2 -> True) difftab (createSingletonRTable $ createNullRTuple $ (getColumnNamesfromRTab tab))
+                            in iJ (\t1 t2 -> True) difftab (createSingletonRTable $ createNullRTuple $ (getColumnNamesFromRTab tab))
                         -- debug
                         -- !dummy2 = trace ("unionSndPart :\n" ++ show unionSndPart) True
                         -- !dummy3 = trace ("u unionFstPart unionSndPart :\n" ++ (show $ u unionFstPart unionSndPart)) True
@@ -2667,9 +2976,9 @@         -- we need to construct the 2nd part of the union with leftTab columns unchanged and rightTab changed        
         unionSndPartTemp1 = d (rJ jpred leftRTab rightRTab) (iJ (flip jpred) rightRTab leftRTab)        
         -- isolate the columns of the rightTab
-        unionSndPartTemp2 = p (getColumnNamesfromRTab rightRTab) unionSndPartTemp1
+        unionSndPartTemp2 = p (getColumnNamesFromRTab rightRTab) unionSndPartTemp1
         -- this join is a trick in order to change names of the rightTab
-        unionSndPart = iJ (\t1 t2 -> True) (createSingletonRTable $ createNullRTuple $ (getColumnNamesfromRTab leftRTab)) unionSndPartTemp2
+        unionSndPart = iJ (\t1 t2 -> True) (createSingletonRTable $ createNullRTuple $ (getColumnNamesFromRTab leftRTab)) unionSndPartTemp2
     in unionFstPart `u` unionSndPart
 
 -- | RTable Union Operator. A short name for the 'runUnion' function
@@ -2691,7 +3000,8 @@     in  V.fromList resultLs
 -}
 
--- | Implements the union of two RTables as a union of two lists (see 'Data.List').
+-- | Implements the union of two RTables.
+-- Note that dublicate 'RTuple' elimination takes places.
 runUnion :: 
     RTable
     -> RTable
@@ -2706,19 +3016,47 @@                 else
                     if isRTabEmpty rt2
                         then rt1
-                        else  
-                            -- construct the union result by concatenating the left table with the subset of tuple from the right table that do
-                            -- not appear in the left table (i.e, remove dublicates)
-                            rt1 V.++ (V.foldr (un) emptyRTable rt2)
-                            where
-                                un :: RTuple -> RTable -> RTable
-                                un tupRight acc = 
-                                    -- Can we find tupRight in the left table?            
-                                    if didYouFindIt tupRight rt1 
-                                        then acc   -- then discard tuplRight ,leave result unchanged          
-                                        else V.snoc acc tupRight  -- else insert tupRight into final result
-
+                        else 
+                            -- check similarity of rtable structures 
+                            if rtabsSameStructure rt1 rt2  
+                                then
+                                    -- run the union
+                                    -- construct the union result by concatenating the left table with the subset of tuple from the right table that do
+                                    -- not appear in the left table (i.e, remove dublicates)
+                                    rt1 V.++ (V.foldr (un) emptyRTable rt2)
+                                else 
+                                   throw $ ConflictingRTableStructures "Cannot run: Union, due to conflicting RTable structures." 
+    where
+        un :: RTuple -> RTable -> RTable
+        un tupRight acc = 
+            -- Can we find tupRight in the left table?            
+            if didYouFindIt tupRight rt1 
+                then acc   -- then discard tuplRight ,leave result unchanged          
+                else V.snoc acc tupRight  -- else insert tupRight into final result
 
+-- | Implements the union-all of two RTables. I.e., a union without dublicate 'RTuple' elimination. Runs in O(m+n).
+runUnionAll :: 
+    RTable
+    -> RTable
+    -> RTable
+runUnionAll rt1 rt2 =
+    if isRTabEmpty rt1 && isRTabEmpty rt2
+        then 
+            emptyRTable
+        else
+            if isRTabEmpty rt1
+                then rt2 
+                else
+                    if isRTabEmpty rt2
+                        then rt1
+                        else  
+                            -- check similarity of rtable structures
+                            if rtabsSameStructure rt1 rt2  
+                                then
+                                    -- run the union
+                                    rt1 V.++ rt2  -- Data.Vector concatenation O(n+m)
+                                else 
+                                   throw $ ConflictingRTableStructures "Cannot run: UnionAll, due to conflicting RTable structures."                             
 
 -- | RTable Intersection Operator. A short name for the 'runIntersect' function
 i = runIntersect
@@ -2733,15 +3071,21 @@         then
             emptyRTable
         else 
-            -- construct the intersect result by traversing the left table and checking if each tuple exists in the right table
-            V.foldr (intsect) emptyRTable rt1
-            where
-                intsect :: RTuple -> RTable -> RTable
-                intsect tupLeft acc = 
-                    -- Can we find tupLeft in the right table?            
-                    if didYouFindIt tupLeft rt2 
-                        then V.snoc acc tupLeft  -- then insert tupLeft into final result
-                        else acc  -- else discard tuplLeft ,leave result unchanged          
+            -- check similarity of rtable structures
+            if rtabsSameStructure rt1 rt2  
+                then
+                    -- run the intersection
+                    -- construct the intersect result by traversing the left table and checking if each tuple exists in the right table
+                    V.foldr (intsect) emptyRTable rt1
+                else 
+                   throw $ ConflictingRTableStructures "Cannot run: Intersect, due to conflicting RTable structures." 
+    where
+        intsect :: RTuple -> RTable -> RTable
+        intsect tupLeft acc = 
+            -- Can we find tupLeft in the right table?            
+            if didYouFindIt tupLeft rt2 
+                then V.snoc acc tupLeft  -- then insert tupLeft into final result
+                else acc  -- else discard tuplLeft ,leave result unchanged          
 
 {-
 runIntersect rt1 rt2 = 
@@ -2785,15 +3129,21 @@                 then
                     rt1
                 else  
-                    -- construct the diff result by traversing the left table and checking if each tuple exists in the right table
-                    V.foldr (diff) emptyRTable rt1
-                    where
-                        diff :: RTuple -> RTable -> RTable
-                        diff tupLeft acc = 
-                            -- Can we find tupLeft in the right table?            
-                            if didYouFindIt tupLeft rt2 
-                                then acc  -- then discard tuplLeft ,leave result unchanged
-                                else V.snoc acc tupLeft  -- else insert tupLeft into final result
+                    -- check similarity of rtable structures
+                    if rtabsSameStructure rt1 rt2  
+                        then
+                            -- run the minus
+                            -- construct the diff result by traversing the left table and checking if each tuple exists in the right table
+                            V.foldr (diff) emptyRTable rt1
+                        else 
+                           throw $ ConflictingRTableStructures "Cannot run: Minus, due to conflicting RTable structures."                             
+    where
+        diff :: RTuple -> RTable -> RTable
+        diff tupLeft acc = 
+            -- Can we find tupLeft in the right table?            
+            if didYouFindIt tupLeft rt2 
+                then acc  -- then discard tuplLeft ,leave result unchanged
+                else V.snoc acc tupLeft  -- else insert tupLeft into final result
 
 -- Important Note:
 -- we need to implement are own equality comparison function "areTheyEqual" and not rely on the instance of Eq defined for RDataType above
@@ -3150,18 +3500,309 @@ 
 -- * ########## RTable DML Operations ##############
 
-
 -- | O(n) append an RTuple to an RTable
+-- Please note that this is an __immutable__ implementation of an 'RTable' insert.
+-- This simply means that the insert operation returns a new 'RTable' and does not
+-- affect the original 'RTable'.
 insertAppendRTab :: RTuple -> RTable -> RTable
-insertAppendRTab rtup rtab = V.snoc rtab rtup
+insertAppendRTab rtup rtab = -- V.snoc rtab rtup
+    if isRTabEmpty rtab && isRTupEmpty rtup 
+        then emptyRTable
+        else
+            if isRTabEmpty rtab && not(isRTupEmpty rtup)
+                then
+                    createSingletonRTable rtup 
+                else
+                    if not(isRTabEmpty rtab) && isRTupEmpty rtup 
+                        then rtab 
+                        else  -- non of the two is empty
+                            -- check similarity of structure vefore the insert
+                            if rtabsSameStructure (createSingletonRTable rtup) rtab
+                                then 
+                                    V.snoc rtab rtup
+                                else 
+                                    throw $ ConflictingRTableStructures "Cannot run: Insert Into Values (insertAppendRTab), due to conflicting RTable structures." 
 
 -- | O(n) prepend an RTuple to an RTable
+-- Please note that this is an __immutable__ implementation of an 'RTable' insert.
+-- This simply means that the insert operation returns a new 'RTable' and does not
+-- affect the original 'RTable'.
 insertPrependRTab :: RTuple -> RTable -> RTable
-insertPrependRTab rtup rtab = V.cons rtup rtab  
+insertPrependRTab rtup rtab = 
+    if isRTabEmpty rtab && isRTupEmpty rtup 
+        then emptyRTable
+        else
+            if isRTabEmpty rtab && not(isRTupEmpty rtup)
+                then
+                    createSingletonRTable rtup 
+                else
+                    if not(isRTabEmpty rtab) && isRTupEmpty rtup 
+                        then rtab 
+                        else  -- non of the two is empty    
+                            -- check similarity of structure vefore the insert
+                            if rtabsSameStructure (createSingletonRTable rtup) rtab
+                                then 
+                                    V.cons rtup rtab 
+                                else 
+                                    throw $ ConflictingRTableStructures "Cannot run: insertPrependRTab, due to conflicting RTable structures." 
 
--- | Update an RTable. Input includes a list of (ColumnName, new Value) pairs.
--- Also a filter predicate is specified in order to restrict the update only to those
--- rtuples that fulfill the predicate
+
+-- | Insert an 'RTable' to an existing 'RTable'. This is equivalent to an @INSERT INTO SELECT@ caluse in SQL.
+-- We want to insert into an 'RTable' the results of a \"subquery\", which in our case is materialized via the
+-- input 'RTable'.
+-- Please note that this is an __immutable__ implementation of an 'RTable' insert.
+-- This simply means that the insert operation returns a new 'RTable' and does not
+-- affect the original 'RTable'.
+-- Also note that the source and target 'RTable's should have the same structure.
+-- By \"structure\", we mean that the 'ColumnName's and the corresponding data types must match. Essentially what we record in the 'ColumnInfo'
+-- must be the same for the two 'RTable's. Otherwise a 'ConflictingRTableStructures' exception will be thrown.
+insertRTabToRTab ::
+                RTable -- ^ Source 'RTable' to be inserted
+                -> RTable -- ^ Target 'RTable'
+                -> RTable -- ^ Final Result
+insertRTabToRTab src trg = 
+    if isRTabEmpty src
+        then trg
+        else
+            if isRTabEmpty trg
+                then src 
+                else -- both src and trg are not empty
+                    
+                    -- check that both rtables have the same structure: 
+                    --      num of columns, column data types and column names.
+                    if rtabsSameStructure src trg 
+                        then
+                            -- run the insert (as a union all)
+                            runUnionAll src trg
+                        else 
+                           throw $ ConflictingRTableStructures "Cannot run: Insert Into <TAB> RTuples, due to conflicting RTable structures." 
+
+-- | Upsert (Update+Insert, aka Merge) Operation. We provide a source 'RTable' and a matching condition ('RUpsertPredicate') to the 'RTuple's 
+-- of the target 'RTable'. An 'RTuple' from the target 'RTable' might match to a single only 'RTuple' in the source 'RTable', or not match at all. 
+-- If it is matched to more than one 'RTuple's then an exception ('UniquenessViolationInUpsert') is thrown. 
+-- When an 'RTuple' from the target 'RTable' is matched to a source 'RTuple', then the corresponding columns of the target 'RTuple' are updated
+-- with the new values provided in the source 'RTuple'. This takes place for the target 'RTuple's that match but also that satisfy the input 
+-- 'RPredicate'. Thus we can restrict further with a filter the 'RTuple's of the target 'RTable' where the update will take place.
+-- Finally, the source 'RTuple's that did not match to the target 'RTable', are inserted (appended) to the target 'RTable'
+--
+-- Please note that this is an __immutable__ implementation of an 'RTable' upsert.
+-- This simply means that the upsert operation returns a new 'RTable' and does not
+-- affect the original 'RTable'.
+-- Moreover, if we have multiple threads updating an 'RTable', due to immutability, each thread \"sees\" its own copy of
+-- the 'RTable' and thus there is no need for locking the updated 'RTuple's, as happens in a common RDBMS.
+--
+-- Also note that the source and target 'RTable's should have the same structure.
+-- By \"structure\", we mean that the 'ColumnName's and the corresponding data types must match. Essentially what we record in the 'ColumnInfo'
+-- must be the same for the two 'RTable's. Otherwise a 'ConflictingRTableStructures' exception will be thrown.
+--
+-- @
+--  An Example:
+--  Source RTable: src = 
+--      Id  |   Msg         | Other
+--      ----|---------------|-------
+--      1   |   "hello2"    |"a"    
+--      2   |   "world2"    |"a"    
+--      3   |   "new"       |"a"    
+--
+--  Target RTable: trg = 
+--      Id  |   Msg         | Other
+--      ----|---------------|-------
+--      1   |   "hello1"    |"b"    
+--      2   |   "world1"    |"b"    
+--      4   |   "old"       |"b"    
+--      5   |   "hello"     |"b"    
+--
+--  >>> upsertRTab  src
+--                  RUpsertPredicate {matchCols = [\"Id\"], matchPred = \\t1 t2 -> t1 \<!\> \"Id\" == t2 \<!\> \"Id\" }
+--                  [\"Msg\"]
+--                  (\\t ->   let 
+--                              msg = case toText (t \<!\> \"Msg\") of
+--                                          Just t -> t
+--                                          Nothing -> pack ""
+--                          in (take 5 msg) == (pack "hello")
+--                  )  -- Msg like "hello%"
+--                  trg
+--
+--  Result RTable: rslt = 
+--      Id  |   Msg         | Other
+--      ----|---------------|-------
+--      1   |   "hello2"    |"b"   (Note that only column \"Msg\" has been overwritten, as per the 3rd argument) 
+--      2   |   "world1"    |"b"    
+--      3   |   "new"       |"a"    
+--      4   |   "old"       |"b"    
+--      5   |   "hello"     |"b"    
+-- @
+--
+upsertRTab ::
+    RTable -- ^ Source 'RTable', i.e., the equivalent to an SQL @USING@ subclause 
+    -> RUpsertPredicate -- ^ The 'RTuple' matching predicate for the merge operation
+
+    -> [ColumnName] -- ^ List of column names to be updated with the corresponding new values coming from the source 'RTuple's 
+                    -- that match with the target 'RTuple's based on the 'RUpsertPredicate' 
+    -> RPredicate  -- ^ A filter that specifies the target 'RTuple's to be updated
+    -> RTable -- ^ The target 'RTable'
+    -> RTable -- ^ Final Result
+upsertRTab srcTab upsPred cols fpred trgTab = 
+    {-
+        README PLEASE: Upsert Algorithm
+
+        upsertRTab srcTab upsPred cols fpred trgTab = 
+
+            insertRTabToRTab (Table S1) $ UNION (Table T1) (Table T2) (Table T3) 
+
+            where
+                Table T1 =  let 
+                                tab  = p (cols ++ (matchCols upsPred)) $ srcTab <semi-join> (f fpred trgTab) <on> (matchPred upsPred) 
+                            
+                                -- this projection will ensure that the right columns will be overwritten
+                                -- dont forget that srcTab and trgTab have the same structure, thus also the same column names 
+                            in  p (getColumnNamesFromRTab trgTab) $ tab <inner-join> (f fpred trgTab) <on> (matchPred upsPred) 
+ 
+                        --      1   |   "hello2"    |"a" 
+
+                Table T2 = (f fpred trgTab) <anti-join> srcTab <on> (matchPred upsPred)
+                        --      5   |   "hello"     |"b"    
+
+                Table T3 = f (not . fpred) trgTab
+                        --      2   |   "world1"    |"b"    
+                        --      4   |   "old"       |"b"    
+
+                Table S1 = srcTab <anti-join> trgTab <on> (matchPred upsPred)
+                        --      3   |   "new"       |"a"    
+        
+        Also, Table T1 must be unique if we group by the columns participating in upsPred (i.e., matchCols upsPred)
+
+    -}
+    if isRTabEmpty srcTab
+        then trgTab
+        else
+            if isRTabEmpty trgTab
+                then srcTab 
+                else -- both src and trg are not empty
+                    
+                    -- check that both rtables have the same structure: 
+                    --      num of columns, column data types and column names.
+                    if rtabsSameStructure srcTab trgTab 
+                        then
+                            -- check uniqueness condition at srcTab: group by the matching columns and make sure there are no dublicates
+                            if not $ isRTabEmpty $
+                                            f (\t -> t <!> "numOfRows" > 1) $
+                                            rG  (\t1 t2 -> Data.List.foldr (\col acc -> (t1 <!> col == t2 <!> col) && acc) True (matchCols upsPred))
+                                                [raggCountStar "numOfRows"]
+                                                (matchCols upsPred)
+                                                srcTab
+                                then
+                                    throw $ UniquenessViolationInUpsert "Cannot run: Upsert because the source RTable is not unique in the matching columns." 
+                                else
+                                    -- run the upsert
+                                    let
+                                        t1 =    let
+                                                    tab  = p (cols ++ (matchCols upsPred)) $ sJ (matchPred upsPred) srcTab (f fpred trgTab)
+                                                    -- debug
+                                                    -- !dummy1 = trace ("tab:\n" ++ show tab) True
+
+                                                    -- this projection will ensure that the right columns will be overwritten
+                                                    -- dont forget that srcTab and trgTab have the same structure, thus also the same column names 
+                                                in  p (getColumnNamesFromRTab trgTab) $ iJ (matchPred upsPred) tab (f fpred trgTab)  
+                                        
+                                        -- debug
+                                        -- !dummy2 = trace ("t1:\n" ++ show t1) True
+
+                                        t2 =  aJ (matchPred upsPred) (f fpred trgTab) srcTab
+
+                                        -- debug
+                                        -- !dummy3 = trace ("t2:\n" ++ show t2) True                                        
+
+                                        t3 = f (not . fpred) trgTab
+
+                                        s1 = aJ (matchPred upsPred) srcTab trgTab 
+
+                                    in insertRTabToRTab s1 $ u t1 $ u t2 t3  
+                        else 
+                           throw $ ConflictingRTableStructures "Cannot run: Upsert due to conflicting RTable structures." 
+
+    
+
+-- | Compares the structure of the input 'RTable's and returns 'True' if these are the same.
+-- By \"structure\", we mean that the 'ColumnName's and the corresponding data types must match. Essentially what we record in the 'ColumnInfo'
+-- must be the same for the two 'RTable's.
+-- Note that in the case of two columns having the same name but one of the two (or both) have a 'dtype' equal to 'UknownType', then
+-- this function assumes that they are the same (i.e., equal 'ColumnInfo's).
+rtabsSameStructure :: RTable -> RTable -> Bool
+rtabsSameStructure rtab1 rtab2 =
+    let 
+        cinfo_list1 = getColumnInfoFromRTab rtab1
+        cinfo_list2 = getColumnInfoFromRTab rtab2
+        -- in the case of two columns with the same name but with one of the two having a Null value,
+        -- then the comparison of the column data types will fail, since the column with the Null values
+        -- will have an "UknownType" in ColumnDType.
+        -- So, we have to normalize the list for this case, so that they pass the equality test
+        --
+        -- Traverse the list, and if you find a column of UknownType, then make it the same type with 
+        -- that of the column with the same name from the other list (if there is such a column)
+        cinfo_list1_new = Data.List.map (normalizeColInfoList cinfo_list2) cinfo_list1
+        cinfo_list2_new = Data.List.map (normalizeColInfoList cinfo_list1) cinfo_list2
+    in  
+        -- In order for the two lists to have the same elements (regardless of their order),
+        -- then the double minus must result to an empty list.
+        ( (cinfo_list1_new \\ cinfo_list2_new) == [] )
+                    &&
+        ( (cinfo_list2_new \\ cinfo_list1_new) == [] )
+    where
+        -- Traverse the list, and if you find a column of UknownType, then make it the same type with 
+        -- that of the column with the same name from the other list (if there is such a column)        
+        normalizeColInfoList :: [ColumnInfo] -> ColumnInfo -> ColumnInfo
+        normalizeColInfoList cinfo_list2 ci1 =  --  (\ci1 -> 
+            -- check if there is a column in the other list with the same name
+            case Data.List.find (\ci2 -> (name ci1) == (name ci2)) cinfo_list2 of
+                Nothing -> ci1  -- do nothing
+                Just ci ->  -- if the type of the 1st one is Uknown
+                            if (dtype ci1) == UknownType
+                                then
+                                    -- change the type to that of ci2
+                                    ColumnInfo {name = (name ci1), dtype = (dtype ci)}
+                                else -- do nothing
+                                    ci1 
+        -- )
+
+-- | Compares the structure of the input 'RTuple's and returns 'True' if these are the same.
+-- By \"structure\", we mean that the 'ColumnName's and the corresponding data types must match. Essentially what we record in the 'ColumnInfo'
+-- must be the same for the two 'RTuple's
+rtuplesSameStructure :: RTuple -> RTuple -> Bool
+rtuplesSameStructure t1 t2 =
+    let 
+        cinfo_list1 = getColumnInfoFromRTuple t1
+        cinfo_list2 = getColumnInfoFromRTuple t2
+    in  
+        -- In order for the two lists to have the same elements (regardless of their order),
+        -- then the double minus must result to an empty list.
+        ( (cinfo_list1 \\ cinfo_list2) == [] )
+                    &&
+        ( (cinfo_list2 \\ cinfo_list1) == [] )
+
+
+
+-- | Delete 'RTuple's from an 'RTable' based on an 'RPredicate'.
+-- Please note that this is an __immutable__ implementation of an 'RTable' update. This simply means that
+-- the delete operation returns a new 'RTable'. So, the original 'RTable' remains unchanged and no deletion in-place
+-- takes place whatsoever.
+-- Moreover, if we have multiple threads deleting an 'RTable', due to immutability, each thread \"sees\" its own copy of
+-- the 'RTable' and thus there is no need for locking the deleted 'RTuple's, as happens in a common RDBMS.
+deleteRTab ::
+        RPredicate  -- ^ Predicate specifying the 'Rtuple's that must be deleted
+    ->  RTable      -- ^ 'RTable' that the deletion will be applied
+    ->  RTable      -- ^ Result 'RTable'
+deleteRTab rpred rtab = f (not . rpred) rtab -- simply omit the rtuples satisfying the predicates    
+
+-- | Update an RTable. The input includes a list of (ColumnName, new Value) pairs.
+-- Also a filter predicate is specified, in order to restrict the update only to those
+-- 'RTuple's that fulfill the predicate.
+-- Please note that this is an __immutable__ implementation of an 'RTable' update. This simply means that
+-- the update operation returns a new 'RTable' that includes all the 'RTuple's of the original 'RTable', both the ones
+-- that have been updated and the others that have not. So, the original 'RTable' remains unchanged and no update in-place
+-- takes place whatsoever.
+-- Moreover, if we have multiple threads updating an 'RTable', due to immutability, each thread \"sees\" its own copy of
+-- the 'RTable' and thus there is no need for locking the updated 'RTuple's, as happens in a common RDBMS.
 updateRTab ::
         [(ColumnName, RDataType)] -- ^ List of column names to be updated with the corresponding new values
     ->  RPredicate  -- ^ An RTuple -> Bool function that specifies the RTuples to be updated
@@ -3169,7 +3810,7 @@     ->  RTable       -- ^ Output RTable
 updateRTab [] _ inputRtab = inputRtab 
 updateRTab ((colName, newVal) : rest) rpred inputRtab = 
-    {-
+    {-  -- READ ME PLEASE --
         Here is the update algorithm:
 
         FinalRTable = UNION (Table A) (Table B)
@@ -3183,10 +3824,34 @@         then emptyRTable
         else
             let 
-                tabA = rtabMap (upsertRTuple colName newVal) (f rpred inputRtab)
+                tabA = rtabMap (updateRTuple colName newVal) (f rpred inputRtab)
                 tabB = f (not . rpred) inputRtab 
             in updateRTab rest rpred (u tabA tabB)
 
+-- | Update an RTuple at a specific column specified by name with a value. If the 'ColumnName'
+-- exists, then the value is updated with the input value. If the 'ColumnName' does not exist,
+-- then a 'ColumnDoesNotExist' exception is thrown.
+updateRTuple ::
+           ColumnName  -- ^ key where the update will take place
+        -> RDataType   -- ^ new value
+        -> RTuple      -- ^ input RTuple
+        -> RTuple      -- ^ output RTuple
+updateRTuple cname newVal tupsrc = 
+    case rtupLookup cname tupsrc of
+        Nothing   ->  throw $ ColumnDoesNotExist cname 
+        _         ->  HM.insert cname newVal tupsrc
+        
+
+-- | Upsert (update/insert) an RTuple at a specific column specified by name with a value
+-- If the cname key is not found then the (columnName, value) pair is inserted. If it exists
+-- then the value is updated with the input value.
+upsertRTuple ::
+           ColumnName  -- ^ key where the upsert will take place
+        -> RDataType   -- ^ new value
+        -> RTuple      -- ^ input RTuple
+        -> RTuple      -- ^ output RTuple
+upsertRTuple cname newVal tupsrc = HM.insert cname newVal tupsrc
+
 -- * ########## RTable IO Operations ##############
 
 -- | Basic data type for defining the desired formatting of an 'RTuple' when printing an RTable (see 'printfRTable').
@@ -3357,17 +4022,17 @@             rtup <- rtab
             let ls = Data.List.map (\(c, v) -> (c, RInt $ fromIntegral $ Data.List.length . rdataTypeToString $ v) ) (rtupleToList rtup)
                 -- create an RTuple with the column names lengths
-                headerLengths = Data.List.zip (getColumnNamesfromRTab rtab) (Data.List.map (\c -> RInt $ fromIntegral $ Data.List.length (T.unpack c)) (getColumnNamesfromRTab rtab))
+                headerLengths = Data.List.zip (getColumnNamesFromRTab rtab) (Data.List.map (\c -> RInt $ fromIntegral $ Data.List.length (T.unpack c)) (getColumnNamesFromRTab rtab))
             -- append  to the rtable also the tuple corresponding to the header (i.e., the values will be the names of the column) in order
             -- to count them also in the width calculation
-            (return $ createRtuple ls) V.++ (return $ createRtuple headerLengths)
+            (return $ createRTuple ls) V.++ (return $ createRTuple headerLengths)
 
         -- Get the max length for each column
         resultRTab = findMaxLengthperColumn lengthRTab
                 where
                     findMaxLengthperColumn :: RTable -> RTable
                     findMaxLengthperColumn rt = 
-                        let colNames = (getColumnNamesfromRTab rt) -- [ColumnName]
+                        let colNames = (getColumnNamesFromRTab rt) -- [ColumnName]
                             aggOpsList = Data.List.map (\c -> raggMax c c) colNames  -- [AggOp]
                         in runAggregation aggOpsList rt
          -- get the RTuple with the results
@@ -3392,17 +4057,17 @@             rtup <- rtab
             let ls = Data.List.map (\(c, v) -> (c, RInt $ fromIntegral $ Data.List.length . rdataTypeToString $ v) ) (rtupleToList rtup)
                 -- create an RTuple with the column names lengths
-                headerLengths = Data.List.zip (getColumnNamesfromRTab rtab) (Data.List.map (\c -> RInt $ fromIntegral $ Data.List.length (T.unpack c)) (getColumnNamesfromRTab rtab))
+                headerLengths = Data.List.zip (getColumnNamesFromRTab rtab) (Data.List.map (\c -> RInt $ fromIntegral $ Data.List.length (T.unpack c)) (getColumnNamesFromRTab rtab))
             -- append  to the rtable also the tuple corresponding to the header (i.e., the values will be the names of the column) in order
             -- to count them also in the width calculation
-            (return $ createRtuple ls) V.++ (return $ createRtuple headerLengths)
+            (return $ createRTuple ls) V.++ (return $ createRTuple headerLengths)
 
         -- Get the max length for each column
         resultRTab = findMaxLengthperColumn lengthRTab
                 where
                     findMaxLengthperColumn :: RTable -> RTable
                     findMaxLengthperColumn rt = 
-                        let colNames = (getColumnNamesfromRTab rt) -- [ColumnName]
+                        let colNames = (getColumnNamesFromRTab rt) -- [ColumnName]
                             aggOpsList = Data.List.map (\c -> raggMax c c) colNames  -- [AggOp]
                         in runAggregation aggOpsList rt
 
@@ -3411,8 +4076,8 @@         julexpr =  EtlMapStart
                 -- Turn each value to an (RInt i) that correposnd to the length of the String representation of the value
                 :-> (EtlC $ 
-                        Source (getColumnNamesfromRTab rtab)
-                        Target (getColumnNamesfromRTab rtab)
+                        Source (getColumnNamesFromRTab rtab)
+                        Target (getColumnNamesFromRTab rtab)
                         By (\[value] -> [RInt $ Data.List.length . rdataTypeToString $ value] )
                         (On $ Tab rtab) 
                         RemoveSrc $
@@ -3426,7 +4091,7 @@                 where
                     findMaxLengthperColumn :: RTable -> RTable
                     findMaxLengthperColumn rt = 
-                        let colNames = (getColumnNamesfromRTab rt) -- [ColumnName]
+                        let colNames = (getColumnNamesFromRTab rt) -- [ColumnName]
                             aggOpsList = V.map (\c -> raggMax c c) colNames  -- [AggOp]
                         in runAggregation aggOpsList rt
 
@@ -3473,7 +4138,7 @@                                 then
                                     colSelectList rtupFmt
                                 else
-                                    getColumnNamesfromRTab rtab -- [ColumnName] 
+                                    getColumnNamesFromRTab rtab -- [ColumnName] 
         listOfLinesCont = Data.List.map (\c -> Data.List.take (Data.List.length (T.unpack c)) (repeat ch)) listOfColNames
         --listOfLinesCont = Data.List.map (\c ->  T.replicate (T.length c) (T.singleton ch)) listOfColNames
         formattedLinesCont = Data.List.map (\(w,l) -> addCharacter (w - (Data.List.length l) + spaceSeparatorWidth) ch l) (Data.List.zip widths listOfLinesCont)
@@ -3488,7 +4153,7 @@     -> RTable
     -> IO ()
 printContLine widths ch rtab = do
-    let listOfColNames =  getColumnNamesfromRTab rtab -- [ColumnName] 
+    let listOfColNames =  getColumnNamesFromRTab rtab -- [ColumnName] 
         listOfLinesCont = Data.List.map (\c -> Data.List.take (Data.List.length (T.unpack c)) (repeat ch)) listOfColNames
         formattedLinesCont = Data.List.map (\(w,l) -> addCharacter (w - (Data.List.length l) + spaceSeparatorWidth) ch l) (Data.List.zip widths listOfLinesCont)
         formattedRowOfLinesCont = Data.List.foldr (\line accum -> line ++ accum) "" formattedLinesCont
@@ -3503,7 +4168,7 @@     -> RTable
     -> IO ()
 printRTableHeaderFmt rtupFmt widths rtab = do -- undefined    
-        let listOfColNames = if rtupFmt /= genRTupleFormatDefault then colSelectList rtupFmt else  getColumnNamesfromRTab rtab -- [ColumnName]
+        let listOfColNames = if rtupFmt /= genRTupleFormatDefault then colSelectList rtupFmt else  getColumnNamesFromRTab rtab -- [ColumnName]
             -- format each column name according the input width and return a list of Boxes [Box]
             -- formattedList =  Data.List.map (\(w,c) -> BX.para BX.left (w + spaceSeparatorWidth) c) (Data.List.zip widths listOfColNames)    -- listOfColNames   -- map (\c -> BX.render . BX.text $ c) listOfColNames
             formattedList = Data.List.map (\(w,c) -> addSpace (w - (Data.List.length (T.unpack c)) + spaceSeparatorWidth) (T.unpack c)) (Data.List.zip widths listOfColNames) 
@@ -3536,7 +4201,7 @@     -> RTable
     -> IO ()
 printRTableHeader widths rtab = do -- undefined    
-        let listOfColNames =  getColumnNamesfromRTab rtab -- [ColumnName]
+        let listOfColNames =  getColumnNamesFromRTab rtab -- [ColumnName]
             -- format each column name according the input width and return a list of Boxes [Box]
             -- formattedList =  Data.List.map (\(w,c) -> BX.para BX.left (w + spaceSeparatorWidth) c) (Data.List.zip widths listOfColNames)    -- listOfColNames   -- map (\c -> BX.render . BX.text $ c) listOfColNames
             formattedList = Data.List.map (\(w,c) -> addSpace (w - (Data.List.length (T.unpack c)) + spaceSeparatorWidth) (T.unpack c)) (Data.List.zip widths listOfColNames) 
@@ -3720,7 +4385,23 @@ -- instance Exception RTimestampFormatLengthMismatch
 
 -- | One (or both) of the input 'String's to function 'toRTimestamp' are empty
-data EmptyInputStringsInToRTimestamp = EmptyInputStringsInToRTimestamp String String deriving(Eq,Show)
+data EmptyInputStringsInToRTimestamp = EmptyInputStringsInToRTimestamp String String deriving(Eq, Show)
 instance Exception EmptyInputStringsInToRTimestamp
 
+-- | This exception means that we have tried to do some operation between two 'RTables', which requires that
+-- the structure of the two is the same. e.g., an @Insert Into <TAB> RTuples@, or a @UNION@ or toher set operations. 
+-- By \"structure\", we mean that the 'ColumnName's and the corresponding data types must match. Essentially what we record in the 'ColumnInfo'
+-- must be the same for the two 'RTable's
+data ConflictingRTableStructures = 
+        ConflictingRTableStructures String  -- ^ Error message indicating the operation that failed.
+        deriving(Eq, Show)
+instance Exception ConflictingRTableStructures
 
+-- | This exception means that we have tried an Upsert operation where the source 'RTable' does not have
+-- a unique set of 'Rtuple's if grouped by the columns used in the matching condition.
+-- This simply means that we cannot determine which of the dublicate 'RTuple's in the source 'RTable'
+-- will overwrite the target 'RTable', when the matching condition is satisfied.
+data UniquenessViolationInUpsert =
+        UniquenessViolationInUpsert String -- ^ Error message
+        deriving(Eq, Show)
+instance Exception UniquenessViolationInUpsert

@@ -29,7 +29,7 @@ module RTable.Data.CSV
    (
         -- * The CSV data type
-        CSV
+        CSV (..)
         ,Row
         ,Column                 
         ,CSVOptions(..)
@@ -157,9 +157,11 @@     fromRTable mf rt = MyType (5::Int)-}
 
 -- | Definition of a CSV file.
--- Treating CSV data as opaque byte strings (see 'Csv' type in Cassava library - "Data.Csv": type Csv = Vector Record)
-type CSV = V.Vector Row -- i.e., CV.Csv
+-- Treating CSV data as opaque byte strings
+newtype CSV = CSV {csv :: V.Vector Row}    
 
+-- type CSV = V.Vector Row -- i.e., CV.Csv
+
 -- | CSV data are \"Tabular\" data thus implement the 'RTabular' interface
 instance RTabular CSV where
     toRTable = csvToRTable
@@ -168,11 +170,9 @@ 
 -- | Definition of a CSV Row.
 -- Essentially a Row is just a Vector of ByteString
--- (type Record = Vector Field)
 type Row = V.Vector Column -- i.e., CV.Record
 
--- | Definition of a CSV Record column.
--- (type Field = ByteString)
+-- | Definition of a CSV column.
 type Column = CV.Field   
 
 -- This typeclass instance is required by CV.decodeByName
@@ -219,7 +219,7 @@                             in csv
                 Right csv -> csv            --Right (hdr, csv) -> csv
         --}        
-    return csvResult
+    return $ CSV csvResult
 
 -- | Yes or No sum type
 data YesNo = Yes | No
@@ -259,7 +259,7 @@ 
                                         csvData
 -}
-    return csvResult
+    return $ CSV csvResult
 
 
 
@@ -276,7 +276,7 @@        FilePath  -- ^ the csv file to be created
     -> CSV       -- ^  input 'CSV'
     -> IO()
-writeCSV f csv = do
+writeCSV f (CSV csv) = do
     let csvBS = CV.encode (V.toList csv)
     BL.writeFile f csvBS
 
@@ -292,7 +292,7 @@ printCSV ::
     CSV     -- ^ input 'CSV' to be printed on screen
     -> IO()
-printCSV csv = do
+printCSV (CSV csv) = do
     -- convert each ByteString field to Text
     {--let csvText = V.map (\r -> V.map (decodeUtf32LE) r) csv
     let csvBS = CV.encode (V.toList csvText)--}
@@ -321,7 +321,7 @@     RTableMData 
     -> CSV 
     -> RTable
-csvToRTable m c = 
+csvToRTable m (CSV c) = 
     V.map (row2RTuple m) c
     where
         row2RTuple :: RTableMData -> Row -> RTuple
@@ -393,7 +393,7 @@         -> RTable   -- ^ input RTable
         -> CSV      -- ^ output CSV
 rtableToCSV m t =       
-    (createCSVHeader m) V.++ (V.map (rtuple2row m) t)
+    CSV $ (csv $ createCSVHeader m) V.++ (V.map (rtuple2row m) t)
     where
         rtuple2row :: RTableMData -> RTuple -> Row 
         rtuple2row md rt =
@@ -452,7 +452,7 @@             let listOfColNames = toListColumnName (rtuplemdata md) --Prelude.map (fst) $ (rtuplemdata md) --HM.toList (rtuplemdata md)
                 listOfByteStrings = Prelude.map (\n -> CV.toField n) listOfColNames  
                 headerRow = V.fromList listOfByteStrings
-            in  V.singleton headerRow
+            in  CSV $ V.singleton headerRow
 
 
 -- In order to be able to decode a CSV bytestring into an RTuple,
@@ -498,8 +498,8 @@         return rtup
 --}
 
--- | Necessary instance in order to convert a CSV file column value to an 'RDataType' value.
-instance CV.FromField RDataType where
+-- Necessary instance in order to convert a CSV file column value to an 'RDataType' value.
+{-instance CV.FromField RDataType where
   parseField dt = do
         -- dt is a ByteString (i.e., a Field) representing some value that we have read from the CSV file (we dont know its type)
         -- we need to construct an RDataType from this value and then wrap it into a Parser Monad and return it
@@ -517,7 +517,7 @@ 
         -- wrap the RDataType into a Parser Monad and return it
         pure rdata
-
+-}
 
 {--
     -- #### NOTE ###
@@ -540,7 +540,7 @@        pure rdata
 --}
 
--- | In order to encode an input RTable into a CSV bytestring 
+-- In order to encode an input RTable into a CSV bytestring 
 -- we need to make Rtuple an instance of the ToNamedRecord typeclass and
 -- implement the toNamedRecord function. 
 -- Where:
@@ -567,7 +567,7 @@ --              type Field = ByteString
 -- @
 --
-instance CV.ToField RDataType where
+{-instance CV.ToField RDataType where
     toField rdata = case rdata of
             RInt i      -> encode (i::Integer)
             --RChar c     -> encode (c::Char)
@@ -578,17 +578,10 @@             RDouble d   -> encode (d::Double)
             Null        -> encode (""::String)
             RDate d f   -> encodeUtf8 d -- encode (d::String)
+-}
 
-{--instance CV.ToField RDataType where
-    toField rdata = case rdata of
-            (i::Int)    -> encode (i::Int)
-            -- RText t     -> encode (t::String) 
-            (t::T.Text)     -> encodeUtf8 t -- encodeUtf8 :: Text -> ByteString
-            (d::Double)   -> encode (d::Double)
-            Null        -> encode (""::String)
---}
 
--- | csv2rtable : turn a input CSV to an RTable.
+-- csv2rtable : turn a input CSV to an RTable.
 -- The input CSV will be a ByteString. We assume that the first line is the CSV header,
 -- including the Column Names. The RTable that will be created will have as column names the headers appearing
 -- in the first line of the CSV.
@@ -613,16 +606,16 @@ --              type Field = ByteString
 -- @
 -- See RTable module for these instance
-csv2rtable :: 
+{-csv2rtable :: 
        BL.ByteString  -- ^ input CSV (we asume that this CSV has a header in the 1st line)
     -> RTable         -- ^ output RTable
 csv2rtable csv = 
     case CV.decodeByName csv of
         Left e -> emptyRTable
         Right (h, v) -> v
-
+-}
 
--- | rtable2csv: encode an RTable into a CSV bytestring
+-- rtable2csv: encode an RTable into a CSV bytestring
 -- The first line of the CSV will be the header, which compirses of the column names.
 --
 -- Internally we use CV.encodeByName to achieve this decoding
@@ -659,15 +652,15 @@ --              type Field = ByteString
 -- @
 -- See 'RTable' module for these instance
-rtable2csv ::
+{-rtable2csv ::
        RTable           -- ^ input RTable
     -> BL.ByteString    -- ^ Output ByteString
 rtable2csv rtab = 
     CV.encodeByName (csvHeaderFromRtable rtab) (V.toList rtab)
+-}
 
+
 -- | creates a 'Data.Csv.Header' (as defined in "Data.Csv") from an 'RTable'
---      type Header = Vector Name
---      type Name = ByteString
 csvHeaderFromRtable ::
                  RTable
               -> CV.Header
@@ -677,17 +670,18 @@                 colListPacked = Prelude.map (encode . T.unpack) colList  -- turn it into a list of ByteStrings ([ByteString])
                 header = V.fromList colListPacked
             in header
+
 -- ##################################################
 -- *  Vector oprtations on CSV
 -- ##################################################
 
 -- | O(1) First row
 headCSV :: CSV -> Row
-headCSV = V.head
+headCSV = V.head . csv
 
 -- | O(1) Yield all but the first row without copying. The CSV may not be empty.
 tailCSV :: CSV -> CSV
-tailCSV = V.tail
+tailCSV = CSV . V.tail . csv
 
 
 -- ##################################################
@@ -705,15 +699,20 @@ -- ##################################################
 
 -- | selectNrows: Returns  the first N rows from a CSV file
-selectNrows::
+selectNrows :: 
        Int             -- ^ Number of rows to select
+    -> CSV   -- ^ Input csv 
+    -> CSV   -- ^ Output csv
+selectNrows n icsv = CSV $ V.take n (csv icsv)
+{-selectNrows::
+       Int             -- ^ Number of rows to select
     -> BL.ByteString   -- ^ Input csv 
     -> BL.ByteString   -- ^ Output csv
 selectNrows n csvi = 
     let rtabi = csv2rtable csvi
         rtabo = limit n rtabi -- restrictNrows n rtabi
     in rtable2csv rtabo
-
+-}
 -- | Column projection on an input CSV file where 
 -- desired columns are defined by position (index)
 -- in the CSV.
@@ -721,16 +720,16 @@              [Int]  -- ^ input list of column indexes
           -> CSV    -- ^ input csv
           -> CSV    -- ^ output CSV
-projectByIndex inds icsv = 
-    V.foldr (prj) V.empty icsv
+projectByIndex inds (CSV icsv) = 
+    V.foldr (prj) (CSV $ V.empty) icsv
     where
         prj :: Row -> CSV -> CSV
-        prj row acc = 
+        prj row (CSV acc) = 
             let 
                 -- construct new row including only projected columns
                 newrow = V.fromList $ Data.List.map (\i -> row V.! i) inds
             in -- add new row in result vector
-                V.snoc acc newrow
+                CSV $ V.snoc acc newrow
 
 -- #####  Exceptions Definitions
 

@@ -4,6 +4,8 @@ -}
 
 {-# LANGUAGE OverloadedStrings #-}
+-- :set -XOverloadedStrings
+
 {-# LANGUAGE ViewPatterns #-}  
 --{-# LANGUAGE BangPatterns #-}
 
@@ -58,6 +60,9 @@     C.writeCSV fo csv
     C.printCSV csv
     --}
+
+    -- *** test CSV to RTable conversion                                        
+    putStrLn "*** test CSV to RTable conversion"
     
     --let newcsv = selectNrows (read n) csv
     let rtmdata = T.createRTableMData ( "TestTable", 
@@ -70,12 +75,16 @@                                       ) 
                                         []  -- primary key
                                         []  -- list of unique keys
-        -- *** test CSV to RTable conversion                                        
+
         rtab = C.toRTable rtmdata csv
         rtabNew = T.limit (read n) rtab
         csvNew = C.fromRTable rtmdata rtabNew   
 
-        -- *** test RFilter & RProjection operation
+    -- *** test RFilter & RProjection operation
+    putStrLn "*** test RFilter & RProjection operation"
+
+    let
+
         rtabNew2 =  let
                        myfilter = T.f (\t -> t!"Name" == T.RText {T.rtext = "Karagiannidis"})
                        myprojection = T.p ["Name","MyTime","Number"]
@@ -131,6 +140,8 @@ 
     
     -- *** test Column Mapping
+    putStrLn "*** test Column Mapping"
+
     -- create a new column holding the doubled value from the source column
     let cmap1 = E.RMap1x1 {E.srcCol = "Number", E.removeSrcCol = E.No, E.trgCol = "NewNumber", E.transform1x1 = \x -> 2*x, E.srcRTupleFilter = \_ -> True}
         rtabNew3 = E.runCM cmap1 rtabNew2
@@ -173,7 +184,11 @@     -- C.printCSV csvNew3
 
     -- *** test inner join operation
-    let rtabNew4 = T.iJ (\t1 t2 -> t1!"Number" == t2!"Number") rtabNew3 rtab 
+    putStrLn "*** test inner join operation"
+
+    let rtabNew4 = -- p ["Name", "MyDate", "MyTime", "Number", "DNumber", "NewNumber"] $ 
+                        T.iJ (\t1 t2 -> t1!"Number" == t2!"Number") rtabNew3 rtab 
+        
         rtmdata4 = T.createRTableMData ( "TestTable4", 
                                          [  ("Name", T.Varchar),
                                             ("MyDate", T.Date "DD/MM/YYYY"), 
@@ -186,11 +201,13 @@                                        []  -- primary key
                                        []  -- list of unique keys
     -- Test Julius
+
         rtabNew4_J = E.etl $ evalJulius $
             EtlMapStart
             :-> (EtlR $
                     ROpStart
-                    :.  (Join (TabL rtabNew3) (Tab rtab) $ JoinOn (\t1 t2 -> t1!"Number" == t2!"Number"))
+                    :. (Join (TabL rtabNew3) (Tab rtab) $ JoinOn (\t1 t2 -> t1!"Number" == t2!"Number"))
+                  --  :. (Select ["Name", "MyDate", "MyTime", "Number", "DNumber", "NewNumber"] $ From Previous)
                 )
 
 {-        rtabNew4_J = E.etl $ evalJulius $
@@ -202,10 +219,15 @@         
     writeResult fo "_t4.csv" rtmdata4 rtabNew4
     writeResult fo "_t4_J.csv" rtmdata4 rtabNew4_J
-    putStrLn "TEST INNER JOIN"
+    -- putStrLn "TEST INNER JOIN"
     T.printRTable rtabNew4_J  
+    {-let    
+        rtabNew4 = emptyRTable
+        rtabNew4_J = emptyRTable-}
 
     -- *** Test union, interesection, diff
+    putStrLn "*** Test union, interesection, diff"
+
     let -- change the value in column NewNumber
         cmap2 = E.RMap1x1 {E.srcCol = "NewNumber", E.removeSrcCol = E.No, E.trgCol = "NewNumber", E.transform1x1 = \x -> x + 100, E.srcRTupleFilter = \_ -> True}
         -- now union the two rtables
@@ -220,11 +242,16 @@                                          ]                                       
                                        )   
                                        []  -- primary key
-                                       []  -- list of unique keys
+                                       []  -- list of unique keys    
 
         rtabNew6 = rtabNew5 `T.i` rtabNew4
+
         rtabNew7 =  rtabNew5 `T.d` rtabNew6
 
+    sequence_ [printRTable rtabNew5, printRTable rtabNew6, printRTable rtabNew7]
+
+    let
+
     -- Test Julius
         rtabNew7_J = E.etl $ evalJulius $
             EtlMapStart
@@ -278,7 +305,10 @@                     :.  (MinusP (Tab rtabNew5) (TabL rtabNew6))
                 )
 
--- test Intermediate named results
+    -- *** test Intermediate named results
+    putStrLn "*** test Intermediate named results"
+
+    let
         rtabNew7a_J = E.etl $ evalJulius $ etlXpression
 
         etlXpression = 
@@ -371,6 +401,8 @@     writeResult fo "_t7_J3.csv" rtmdata5 rtabNew7_J3
 
     -- *** Change existing column name 
+    putStrLn "*** Change existing column name"
+
     let -- change the value in column NewNumber
         cmap3 = E.RMap1x1 {E.srcCol = "NewNumber", E.removeSrcCol = E.Yes, E.trgCol = "NewNewNumber", E.transform1x1 = \x -> x, E.srcRTupleFilter = \_ -> True}
         rtabNew8 = E.runCM cmap3 rtabNew5
@@ -402,7 +434,9 @@     writeResult fo "_t8_J.csv" rtmdata8 rtabNew8_J
     T.printRTable rtabNew8_J
 
-    -- Test a RMapNx1 column mapping
+    -- *** Test a RMapNx1 column mapping
+    putStrLn "*** Test a RMapNx1 column mapping"
+
     let 
         cmap4 = E.RMapNx1 {E.srcColGrp = ["Name","MyTime"], E.removeSrcCol = E.Yes, E.trgCol = "New_Nx1_Col", E.transformNx1 = (\[n, T.RTime{T.rtime = t}] -> n `rdtappend` (T.RText "<---->") `rdtappend` (T.rTimestampToRText T.stdTimestampFormat t)), E.srcRTupleFilter = \_ -> True}
         rtabNew9 = E.runCM cmap4 rtabNew8
@@ -434,7 +468,9 @@     writeResult fo "_t9_J.csv" rtmdata9 rtabNew9_J    
     T.printRTable rtabNew9_J    
 
- -- Test a RMap1xN column mapping
+ -- *** Test a RMap1xN column mapping
+    putStrLn "*** Test a RMap1xN column mapping"
+
     let 
         cmap5 = E.RMap1xN {E.srcCol = "New_Nx1_Col", E.removeSrcCol = E.No, E.trgColGrp = ["1xN_A","1xN_B","1xN_C"], E.transform1xN = (\(T.RText txt) -> [T.RText (Data.Text.take 5 txt), T.RText "<-|-|-|->", T.RText (Data.Text.takeEnd 10 txt)]), E.srcRTupleFilter = \_ -> True}
         rtabNew10 = E.runCM cmap5 rtabNew9
@@ -469,7 +505,9 @@     writeResult fo "_t10_J.csv" rtmdata10 rtabNew10_J 
     T.printRTable rtabNew10_J                                
 
--- Test a RMapNxM column mapping
+-- *** Test a RMapNxM column mapping
+    putStrLn "*** Test a RMapNxM column mapping"
+
     let 
         cmap6 = E.RMapNxM {E.srcColGrp = ["1xN_A","1xN_B","1xN_C"], E.removeSrcCol = E.Yes, E.trgColGrp = ["ColNew1","ColNew2"], E.transformNxM = transformation, E.srcRTupleFilter = \_ -> True}
                 where
@@ -507,7 +545,9 @@     writeResult fo "_t11_J.csv" rtmdata11 rtabNew11_J 
     T.printRTable rtabNew11_J                                     
 
--- Test removeColumn operation
+-- *** Test removeColumn operation
+    putStrLn "*** Test removeColumn operation"
+
     let 
         rtabNew12 = T.removeColumn "NewNewNumber" (T.removeColumn "MyDate" rtabNew11)        
         rtmdata12 = T.createRTableMData ( "TestTable12", 
@@ -545,7 +585,9 @@     print rtabNew12_J
     T.printRTable rtabNew12_J
 
---  Test combined Roperations
+--  *** Test combined Roperations
+    putStrLn "*** Test combined Roperations"
+
     let
         myfilter = T.f (\t -> t!"Name" == T.RText {T.rtext = "Karagiannidis"})
         myprojection = T.p ["Name","MyTime","Number", "ColNew2"]
@@ -582,7 +624,9 @@     print rtabNew13_J
     T.printRTable rtabNew13_J
 
--- Test Left Outer Join
+-- *** Test Left Outer Join
+    putStrLn "*** Test Left Outer Join"
+
     let rtabNew14 = T.lJ (\t1 t2 -> t1!"Number" == t2!"Number") rtab rtabNew3
         rtmdata14 = T.createRTableMData ( "TestTable14", 
                                          [  ("Name", T.Varchar),
@@ -600,7 +644,7 @@     -- the first part is the join
     let rtabNew14_dbg_1part = T.iJ (\t1 t2 -> t1!"Number" == t2!"Number") rtab rtabNew3  -- these are the rows of the left tab (enhanced with new columns) that satisfy the join
         -- project only left tab's columns
-        rtabNew14_dbg_1part_proj = T.p (T.getColumnNamesfromRTab rtab) rtabNew14_dbg_1part
+        rtabNew14_dbg_1part_proj = T.p (T.getColumnNamesFromRTab rtab) rtabNew14_dbg_1part
 
 
         -- enhance the left tab with the new columns with NULL values
@@ -616,7 +660,7 @@         --        T.decodeRTable "DNumber" (T.RText "x") T.Null T.Null T.Ignore diffTab
 
         -- enhance this with null columns from the non-preserving table
-        rtabNew14_dbg_2part = T.iJ (\t1 t2 -> True) rtabNew14_dbg_2part_a (T.createSingletonRTable $ T.createNullRTuple $ (T.getColumnNamesfromRTab rtabNew3))
+        rtabNew14_dbg_2part = T.iJ (\t1 t2 -> True) rtabNew14_dbg_2part_a (T.createSingletonRTable $ T.createNullRTuple $ (T.getColumnNamesFromRTab rtabNew3))
 
         -- finally, union the two parts
         rtabNew14_dbg = T.u rtabNew14_dbg_1part rtabNew14_dbg_2part
@@ -641,7 +685,9 @@     putStrLn "Test LEFT JOIN"
     T.printRTable rtabNew14_J
 
--- Test Right Outer Join
+-- *** Test Right Outer Join
+    putStrLn "*** Test Right Outer Join"
+
     let rtabNew15 = T.rJ (\t1 t2 -> t1!"Number" == t2!"Number") rtab rtabNew3
         rtmdata15 = T.createRTableMData ( "TestTable15", 
                                          [  ("Name", T.Varchar),
@@ -668,7 +714,9 @@     putStrLn "Test RIGHT JOIN"
     T.printRTable rtabNew15_J
 
--- Test Full Outer Join
+-- *** Test Full Outer Join
+    putStrLn "*** Test Full Outer Join"
+
     let rtabNew15fo = T.foJ (\t1 t2 -> t1!"Number" == t2!"Number") rtab rtabNew3
         rtabNew15fo2 = T.foJ (\t1 t2 -> t1!"Number" == t2!"Number") rtabNew3 rtab 
         rtmdata15 = T.createRTableMData ( "TestTable15", 
@@ -709,10 +757,12 @@ 
 
 
--- Test Aggregation
+-- *** Test Aggregation
+    putStrLn "*** Test Aggregation"
+
     -- 
     let rtabNew16 = T.rAgg [T.raggSum "Number" "SumNumber" 
-                            , T.raggCount "Number" "CountNumber"
+                            , T.raggCount "Number" "CountNumber"                            
                             , T.raggAvg "Number" "AvgNumber" 
                             , T.raggSum "Name" "SumName"
                             ,T.raggMax "DNumber" "maxDNumber"
@@ -722,6 +772,9 @@                             ,T.raggMin "Number" "minNumber"
                             ,T.raggMin "Name" "minName"
                             ,T.raggAvg "Name" "AvgName"
+                            ,T.raggCountDist "Number" "CountNumberDist"                            
+                            ,T.raggCountDist "Name" "CountNameDist"
+                            ,T.raggCountStar "CountStar"                           
                          ] rtabNew
         rtmdata16 = T.createRTableMData ( "TestTable16", 
                                          [  --("Name", T.Varchar),
@@ -738,6 +791,9 @@                                             ,("minNumber", T.Integer)
                                             ,("minName", T.Varchar)
                                             ,("AvgName", T.Double)
+                                            ,("CountNumberDist", T.Integer)
+                                            ,("CountNameDist", T.Integer)
+                                            ,("CountStar", T.Integer)
                                             --,("NewNumber", T.Integer)
                                          ]                                       
                                        )   
@@ -761,15 +817,22 @@                                     ,Min "DNumber" $ As "minDNumber"
                                     ,Min "Number" $ As "minNumber"
                                     ,Min "Name" $ As "minName"
-                                    ,Avg "Name" $ As "AvgName"] $ From $ Tab rtabNew)
+                                    ,Avg "Name" $ As "AvgName"
+                                    ,CountDist "Number" $ As "CountNumberDist"
+                                    ,CountDist "Name" $ As "CountNameDist"
+                                    ,CountStar $ As "CountStar"
+                                ] $ From $ Tab rtabNew)
                 )
 
     writeResult fo "_t16.csv" rtmdata16 rtabNew16
     writeResult fo "_t16_J.csv" rtmdata16 rtabNew16_J
 
+    T.printRTable rtabNew16
     T.printRTable rtabNew16_J
 
--- Test GroupBy
+-- *** Test GroupBy
+    putStrLn "*** Test GroupBy"
+
     -- 
     let rtabNew17 = T.rG    (\t1 t2 ->  t1!"Name" == t2!"Name" && t1!"MyTime" == t2!"MyTime")  
                             [   T.raggSum "Number" "SumNumber" 
@@ -852,28 +915,29 @@     writeResult fo "_t17.csv" rtmdata17 rtabNew17
     writeResult fo "_t17_J.csv" rtmdata17 rtabNew17_J
 
-    putStrLn "#### TEST GROUP BY ###"
+    -- putStrLn "#### TEST GROUP BY ###"
     T.printRTable rtabNew 
     T.printRTable rtabNew17_J
 
-    -- test groupNoAgg function
-    putStrLn "GroupBy with [] AggOp list"
+    -- *** test groupNoAgg function
+    putStrLn "*** GroupBy with [] AggOp list"
+
     T.printRTable rtabNew17_J2
-    putStrLn "Test groupNoAgg function"
+    putStrLn "*** Test groupNoAgg function"
     T.printRTable $
         groupNoAgg  (\t1 t2 -> t1!"Name" == t2!"Name" && t1!"MyTime" == t2!"MyTime")
                     ["Name", "MyTime"]
                     rtabNew
 
     -- test groupNoAggList function                    
-    putStrLn "Test groupNoAggList function"
+    putStrLn "*** Test groupNoAggList function"
     mapM_ (T.printRTable) $
         groupNoAggList  (\t1 t2 -> t1!"Name" == t2!"Name" && t1!"MyTime" == t2!"MyTime")
                     ["Name", "MyTime"]
                     rtabNew
 
 
-    putStrLn "#### TEST CUSTOM Aggregation ###"
+    putStrLn "*** TEST CUSTOM Aggregation"
     putStrLn "Implement a custom listagg()"
 
     printRTable $ juliusToRTable $
@@ -905,6 +969,227 @@ 
     -- T.printRTable rtabNew18
     T.printfRTable (genRTupleFormat [ "Name", "ListAggName", "CountName"] genDefaultColFormatMap) $ rtabNew18
+
+    putStrLn "*** Test StrAgg aggregation"
+
+    rtabNew19 <- runJulius $
+            EtlMapStart
+            :-> (EtlR $
+                    ROpStart
+                    :.  (GroupBy ["Name"] -- , "MyTime"] 
+                            (AggOn [    {-Sum "Number" $ As "SumNumber"
+                                        ,Count "Name" $ As "CountName"
+                                        --,Avg "Number" $ As "AvgNumber" 
+                                        ,Sum "Name" $ As "SumName"
+                                        ,Count "DNumber" $ As "CountDNumber"
+                                        ,Max "DNumber" $ As "maxDNumber"
+                                        --,Max "Number" $ As "maxNumber"
+                                        ,Max "Name" $ As "maxName"
+                                        ,-}
+                                        Count "Name" $ As "CountName"
+                                        ,StrAgg "Name" (As "ListAggName") ";" 
+                                    ]  $ From $ Tab rtabNew)
+                            $ GroupOn (\t1 t2 ->  t1!"Name" == t2!"Name" )) -- && t1!"MyTime" == t2!"MyTime"))
+                )
+
+    -- T.printRTable rtabNew18
+    T.printfRTable (genRTupleFormat [ "Name", "ListAggName", "CountName"] genDefaultColFormatMap) $ rtabNew19
+
+
+    -- Test updateRTab
+    putStrLn "*** Test RTable update"
+
+    printRTable rtabNew
+    printRTable $ updateRTab [("Name" , RText "Mr. Smith")] (\t -> t <!> "Name" == RText "Karagiannidis") rtabNew
+
+    printRTable $ juliusToRTable $
+            EtlMapStart
+            :-> (EtlR $
+                    ROpStart
+                    :. (Update (Tab rtabNew) (Set [("Name" , RText "Mr. Smith")]) $
+                            FilterBy (\t -> t <!> "Name" == RText "Karagiannidis")
+                        )                    
+                )
+
+    -- Test UnionAll
+    putStrLn "*** Test Union All operation"            
+
+    printRTable $ juliusToRTable $
+            EtlMapStart
+            :-> (EtlR $  -- rtabNew5
+                    ROpStart
+                    :. (UnionAll (TabL rtabNew) (Tab rtabNew))
+                    :. (OrderBy [ ("MyTime",Asc),("DNumber", Asc),("Name",Asc),("MyDate",Asc)
+                                ] $ 
+                            From Previous 
+                    )
+                )
+
+    -- Test insertRTabToRTab
+    putStrLn "*** Test insertRTabToRTab function"  
+
+    printRTable $ insertRTabToRTab rtabNew $ rtabNew
+
+    printRTable $ insertRTabToRTab rtabNew $ f (\t -> t <!> "Name" == RText "Θεοδώρου") rtabNew
+
+    printRTable $ p ["Name"] rtabNew
+    printRTable $ insertRTabToRTab (p ["Name"] rtabNew) $ p ["Name"] rtabNew
+
+    -- Test Insert Into Values clause
+    putStrLn "*** Test Insert Into Values clause"
+
+    printRTable $ juliusToRTable $
+        EtlMapStart
+        :-> (EtlR $
+                ROpStart
+                :.(Insert $
+                        Into (Tab rtabNew) $
+                            Values [    ("DNumber", RDouble 23.89)
+                                        ,("MyTime", RTime $ toRTimestamp "DD/MM/YYYY HH:MI:SS" "31/10/2018 17:24:00" )
+                                        ,("Name", RText "Michael J. Fox")
+                                        ,("MyDate", RDate{rdate = "31/10/2018", dtformat = "DD/MM/YYYY"})
+                                        ,("Number", RInt 58)
+                                    ]
+                )
+            )
+
+    putStrLn "*** Test Insert Into RTuples clause"
+
+    printRTable $ juliusToRTable $
+                        EtlMapStart
+                        :-> (EtlR $
+                                ROpStart
+                                :.(Filter (From $ Tab rtabNew) $ 
+                                        FilterBy (\t -> t <!> "Name" == RText "nkarag")
+                                )
+                            )
+
+    printRTable $ juliusToRTable $
+        EtlMapStart
+        :-> (EtlR $
+                ROpStart
+                :.(Insert $
+                        Into (Tab rtabNew) $
+                            RTuples $ TabSrc $ juliusToRTable $
+                                                EtlMapStart
+                                                :-> (EtlR $
+                                                        ROpStart
+                                                        :.(Filter (From $ Tab rtabNew) $ 
+                                                                FilterBy (\t -> t <!> "Name" == RText "nkarag")
+                                                        )
+                                                    )
+                )
+                :.(OrderBy [("Name", Asc)] $ From Previous)
+            )
+
+    -- Test upsertRTab
+    putStrLn "*** Test Upsert Operation" 
+    
+    let trgTab = addSurrogateKeyJ "SK" (0::Int) rtabNew
+        srcTab = f (\t -> t <!> "Name" == RText "Filippos" || t <!> "Name" == RText "Mr. Smith" ) $ 
+                    insertAppendRTab  
+                        (createRTuple [  ("Name", (RText "Filippos"))
+                                        ,("MyDate", RDate {rdate = "22/12/2018", dtformat = "DD/MM/YYYY"})
+                                        ,("MyTime", RTime $ toRTimestamp "DD/MM/YYYY HH:MI:SS" "22/12/2018 21:22:00") 
+                                        ,("Number", RInt 15)
+                                        ,("DNumber", RDouble 34.67)
+                                        ,("SK", RInt 14)
+                                     ]) $ 
+                            updateRTab [("Name" , RText "Mr. Smith")] (\t -> t <!> "Name" == RText "Karagiannidis") trgTab
+
+    putStrLn "srcTab"
+    printRTable $ runOrderBy [("SK", Asc)] srcTab
+    putStrLn "trgTab"
+    printRTable $ runOrderBy [("SK", Asc)] trgTab
+
+
+    putStrLn "upsertRTab"
+    printRTable $ rO [("SK", Asc)] $
+        upsertRTab  srcTab
+                    (RUpsertPredicate ["SK"] (\t1 t2 -> t1 <!> "SK" == t2 <!> "SK"))
+                    ["Name"]
+                    (\t -> isNull $ t <!> "MyDate")
+                    trgTab
+
+    putStrLn "Test Upsert Julius clause"
+    printRTable $ juliusToRTable $
+        EtlMapStart
+        :-> (EtlR $
+                ROpStart
+                :. (Upsert $
+                        MergeInto (Tab trgTab) $
+                            Using (TabSrc srcTab) $
+                                MergeOn (RUpsertPredicate ["SK"] (\t1 t2 -> t1 <!> "SK" == t2 <!> "SK")) $
+                                    WhenMatchedThen $
+                                        UpdateCols ["Name"] $
+                                            FilterBy (\t -> isNull $ t <!> "MyDate")
+                )
+                :. (OrderBy [("SK", Asc)] $ From Previous)
+            )
+
+    putStrLn "*** Test anti-join and semi-join"
+    putStrLn "trgTab <anti-join> srcTab (t1 <!> \"SK\" == t2 <!> \"SK\")"
+    printRTable $ rO [("SK", Asc)] $ aJ (\t1 t2 -> t1 <!> "SK" == t2 <!> "SK") trgTab srcTab
+
+    putStrLn "AntiJoin Julius clause"
+    printRTable $ juliusToRTable $
+        EtlMapStart
+        :-> (EtlR $
+                ROpStart
+                :.(AntiJoin (TabL trgTab) (Tab srcTab) $ 
+                        JoinOn (\t1 t2 -> t1 <!> "SK" == t2 <!> "SK")
+                )
+                :.(OrderBy [("SK", Asc)] $ From Previous)
+            )
+
+    printRTable $ juliusToRTable $
+        EtlMapStart
+        :-> (EtlR $
+                ROpStart
+                :.(AntiJoinP (Tab trgTab) (TabL srcTab) $ 
+                        JoinOn (\t1 t2 -> t1 <!> "SK" == t2 <!> "SK")
+                )
+                :.(OrderBy [("SK", Asc)] $ From Previous)
+            )
+
+    putStrLn "trgTab <semi-join> srcTab (t1 <!> \"SK\" == t2 <!> \"SK\")"
+    printRTable $ rO [("SK", Asc)] $ sJ (\t1 t2 -> t1 <!> "SK" == t2 <!> "SK") trgTab srcTab
+
+    putStrLn "SemiJoin Julius clause"
+    printRTable $ juliusToRTable $
+        EtlMapStart
+        :-> (EtlR $
+                ROpStart
+                :.(SemiJoin (TabL trgTab) (Tab srcTab) $ 
+                        JoinOn (\t1 t2 -> t1 <!> "SK" == t2 <!> "SK")
+                )
+                :.(OrderBy [("SK", Asc)] $ From Previous)
+            )
+
+    printRTable $ juliusToRTable $
+        EtlMapStart
+        :-> (EtlR $
+                ROpStart
+                :.(SemiJoinP (Tab trgTab) (TabL srcTab) $ 
+                        JoinOn (\t1 t2 -> t1 <!> "SK" == t2 <!> "SK")
+                )
+                :.(OrderBy [("SK", Asc)] $ From Previous)
+            )
+
+    putStrLn "*** Test deleteRTab"
+    printRTable trgTab
+    printRTable $ deleteRTab (\t -> t <!> "SK" > RInt 7) trgTab
+
+    putStrLn "*** Test Delete clause"
+    printRTable $ juliusToRTable $
+        EtlMapStart
+        :-> (EtlR $
+                ROpStart
+                :.(Delete (From $ Tab trgTab) $ 
+                        Where (\t -> t <!> "SK" > RInt 7))
+                :.(Delete (From Previous) $ 
+                        Where (\t -> t <!> "SK" == RInt 3))
+            )
 
 
     where