diff --git a/ChangeLog b/ChangeLog --- a/ChangeLog +++ b/ChangeLog @@ -1,9 +1,17 @@ -2018-03-10 Mihai Giurgeanu +2018-03-15 Mihai Giurgeanu - * stack.yaml: update dependencies in to work with remote repositories + * NEWS: added NEWS file - * Release 0.3.1.0 + * transfer-db.cabal (build-depends): update dependencies on console-program + (version): 0.3.1.1 + * stack.yaml (packages): removed console-program + (extra-deps): add console-program-0.4.2.2 + +2018-03-10 Mihai Giurgeanu + + * stack.yaml: update dependencies to work with remote repositories + 2017-12-29 Mihai Giurgeanu * src/Main.hs (commands): add correctivePlan command @@ -14,7 +22,7 @@ 2017-12-19 Mihai Giurgeanu - * src/TransferDB.hs (transferDB): new functionality to skip a number of batches from the list of batches and to only process a certain count of batches + * src/TransferDB.hs (transferDB): add skip and count functionality * src/Main.hs (runTransferPlan): add count and drop options to run command @@ -26,7 +34,8 @@ 2017-11-26 Mihai Giurgeanu - * lib/Database/TransferDB/Commons.hs (withEnvConnection): add withEnvConnection and withEnvConnection' + * lib/Database/TransferDB/Commons.hs (withEnvConnection) + (withEnvConnection'): add definitions 2017-11-24 Mihai Giurgeanu @@ -34,8 +43,7 @@ 2017-11-07 Mihai Giurgeanu - * src/Main.hs (commands): add dump and restore commands to transfer-db - executable. + * src/Main.hs (commands): add dump and restore commands 2017-11-03 Mihai Giurgeanu @@ -45,10 +53,6 @@ * src/TransferDB.hs: use simple logging framework instead 'hPutStrLn stderr' -2017-10-27 Mihai Giurgeanu - - * (release 0.3.0.0): subcommands in program interface, generate plans - 2017-09-21 Mihai Giurgeanu * (change summary): add functionality to generate plans @@ -57,14 +61,10 @@ * src/Main.hs (main): change program interface to accept commands (run, help) -2017-09-19 Mihai Giurgeanu - - * (release 0.2.0.0): transfer plan, work in restartable batches - 2017-09-18 Mihai Giurgeanu - * src/Main.hs (transferTable): use CLI types instead of ODBC type to avoid problem with PostgreSQL driver for LONGVARBINARY - (transferBufferSize): increase buffer to 64k to support max 64k PostgreSQL text fields (ODBC driver limitation for PostgreSQL) + * src/Main.hs (transferTable): use CLI types instead of ODBC + (transferBufferSize): increase buffer to 64k 2017-09-17 Mihai Giurgeanu @@ -76,6 +76,4 @@ * src/Main.hs (main): change the command line argumets to get the plan file (main): use Options module to process command line arguments (transferDB): read all its arguments from the ReaderT structure TransferOptions - - * (release 0.1.0.0): No batch, "copy all" version of the software diff --git a/NEWS b/NEWS new file mode 100644 --- /dev/null +++ b/NEWS @@ -0,0 +1,63 @@ +transfer-db NEWS - history of user-visible changes +================================================== + +transfer-db-0.3.1.1 +------------------- + +Release date: 2018-03-15 + +Just a minor release, updating the dependencies on console-program package +so transfer-db could be built with cabal. + +Changes: + +* update console-program dependencies +* added NEWS file to keep the history of user visible changes + + +transfer-db-0.3.1.0 +------------------- + +Release date: 2018-03-10 + +Changes: + +* update stack configuration to simplify the build requirements +* add correctivePlan command to generate plans to make corrections to an execution +* add count and drop options to run command (for running plans) +* multithreading parallel proessing of batches +* add dump and restore commands +* a new database agnostic dump library, based on ODBC + +transfer-db-0.3.0.0 +------------------- + +Release date: 2017-10-27 + +Changes: + +* add plan generation functionlity +* modified command line interface to accept commands (run, help) + +transfer-db-0.2.0.0 +------------------- + +Relese date: 2017-09-19 + +Run the database transfer according to a transfer plan. + +Changes: + +* read transfer batch from a transfer plan with options to control + which tables and which data in the tables should be transferred +* commit on every transferred table, so a batch could be restarted + +transfer-db-0.1.0.0 +------------------- + +Release date: 2017-09-15 + +First stable release of transfer-db application and library. It transfers all +the table in the database schema with no options to control what to be trnasferred. + + diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -7,3 +7,60 @@ application is easily portable to other SQL/CLI implementations. It includes a database agnostic dump file format library. + +Current release is 0.3.1.1. + +Release Notes +------------- + +The dump and restore are not finalized yet, so they don't work with the current version. + + +SYNOPSYS +-------- + +To use transfer-db you either right your own transfer plan by hand +or use transfer-db to generate a transfer plan for you. You can +edit the generated tranfer plan. + +A transfer plan is an [yaml](http://yaml.org) file specifying the +database connection parameters and what tables and data should be +transferred. + +Gnereating a plan: + + transfer-db makePlan splitByRows -d -u -p -s -D -U -P -S > my-plan.yaml + +This command will generate on the standard output a plan for transferring the database SOURCEDB +DESTDB in batches of `` rows (the splitByRows sub-command). In case of failure, you could +restart the process by skipping the completed batches. + +After that, simply run: + + transfer-db run my-plan.yaml > therun-0001.log + +This command will run the plan in the `my-plan.yaml` file and will capture the `stdout` of +`transfer-db` command into the file `therun-0001.log`. This file logs the transferred tables +and batches and could be used to generate a corrective plan in case something went wrong. On +console (`stderr`) there will be printed detailed information about running the plan. + +For more advanced options, on controlling the number of threads or of what batches +should be run, type: + + transfer-db help + +To generate a corrective plan, you need the original plan and the `stdout` log generated by +the `run` command: + + transfer-db correctivePlan my-plan.yaml -sfix-001.sql -pnew-plan.yaml -itherun-001.log + +This command will read the original plan from `my-plan.yaml` file, the log of the previous +run from `therun-001.log` file and will generate 2 files: + +* `fix-001.sql` that you will need to run to against the destination database in order + to remove partial batches. +* `new-plan.yaml` that you should use with for a new transfer run, to transfer the remaining + records. + + + diff --git a/transfer-db.cabal b/transfer-db.cabal --- a/transfer-db.cabal +++ b/transfer-db.cabal @@ -1,5 +1,5 @@ name: transfer-db -version: 0.3.1.0 +version: 0.3.1.1 -- synopsis: description: Simple SQL/CLI application that transfers data between 2 databases. homepage: http://hub.darcs.net/mihaigiurgeanu/transfer-db @@ -12,6 +12,7 @@ build-type: Simple cabal-version: >=1.10 extra-source-files: README.md, + NEWS, ChangeLog library @@ -48,7 +49,7 @@ transformers >= 0.5 && < 0.6, yaml >= 0.8.22 && < 0.9, aeson >= 1.0, - console-program >= 0.4.2.0 && < 0.5, + console-program >= 0.4.2.2 && < 0.5, bytestring >= 0.10.8 && < 0.11, logging >= 3.0 && < 4, time >= 1.8 && < 1.9,