transfer-db 0.3.1.0 → 0.3.1.1
raw patch · 4 files changed
+140/−21 lines, 4 filesdep ~console-program
Dependency ranges changed: console-program
Files
- ChangeLog +17/−19
- NEWS +63/−0
- README.md +57/−0
- transfer-db.cabal +3/−2
ChangeLog view
@@ -1,9 +1,17 @@-2018-03-10 Mihai Giurgeanu <mihai.giurgeanu@gmail.com> +2018-03-15 Mihai Giurgeanu <mihai.giurgeanu@gmail.com> - * stack.yaml: update dependencies in to work with remote repositories + * NEWS: added NEWS file - * Release 0.3.1.0 + * transfer-db.cabal (build-depends): update dependencies on console-program + (version): 0.3.1.1 + * stack.yaml (packages): removed console-program + (extra-deps): add console-program-0.4.2.2 + +2018-03-10 Mihai Giurgeanu <mihai.giurgeanu@gmail.com> + + * stack.yaml: update dependencies to work with remote repositories + 2017-12-29 Mihai Giurgeanu <mihai.giurgeanu@gmail.com> * src/Main.hs (commands): add correctivePlan command @@ -14,7 +22,7 @@ 2017-12-19 Mihai Giurgeanu <mihai.giurgeanu@gmail.com> - * src/TransferDB.hs (transferDB): new functionality to skip a number of batches from the list of batches and to only process a certain count of batches + * src/TransferDB.hs (transferDB): add skip and count functionality * src/Main.hs (runTransferPlan): add count and drop options to run command @@ -26,7 +34,8 @@ 2017-11-26 Mihai Giurgeanu <mihai.giurgeanu@gmail.com> - * lib/Database/TransferDB/Commons.hs (withEnvConnection): add withEnvConnection and withEnvConnection' + * lib/Database/TransferDB/Commons.hs (withEnvConnection) + (withEnvConnection'): add definitions 2017-11-24 Mihai Giurgeanu <mihai.giurgeanu@gmail.com> @@ -34,8 +43,7 @@ 2017-11-07 Mihai Giurgeanu <mihai.giurgeanu@gmail.com> - * src/Main.hs (commands): add dump and restore commands to transfer-db - executable. + * src/Main.hs (commands): add dump and restore commands 2017-11-03 Mihai Giurgeanu <mihai.giurgeanu@gmail.com> @@ -45,10 +53,6 @@ * src/TransferDB.hs: use simple logging framework instead 'hPutStrLn stderr' -2017-10-27 Mihai Giurgeanu <mihai.giurgeanu@gmail.com> - - * (release 0.3.0.0): subcommands in program interface, generate plans - 2017-09-21 Mihai Giurgeanu <mihai.giurgeanu@gmail.com> * (change summary): add functionality to generate plans @@ -57,14 +61,10 @@ * src/Main.hs (main): change program interface to accept commands (run, help) -2017-09-19 Mihai Giurgeanu <mihai.giurgeanu@gmail.com> - - * (release 0.2.0.0): transfer plan, work in restartable batches - 2017-09-18 Mihai Giurgeanu <mihai.giurgeanu@gmail.com> - * src/Main.hs (transferTable): use CLI types instead of ODBC type to avoid problem with PostgreSQL driver for LONGVARBINARY - (transferBufferSize): increase buffer to 64k to support max 64k PostgreSQL text fields (ODBC driver limitation for PostgreSQL) + * src/Main.hs (transferTable): use CLI types instead of ODBC + (transferBufferSize): increase buffer to 64k 2017-09-17 Mihai Giurgeanu <mihai.giurgeanu@gmail.com> @@ -76,6 +76,4 @@ * src/Main.hs (main): change the command line argumets to get the plan file (main): use Options module to process command line arguments (transferDB): read all its arguments from the ReaderT structure TransferOptions - - * (release 0.1.0.0): No batch, "copy all" version of the software
+ NEWS view
@@ -0,0 +1,63 @@+transfer-db NEWS - history of user-visible changes+==================================================++transfer-db-0.3.1.1+-------------------++Release date: 2018-03-15++Just a minor release, updating the dependencies on console-program package+so transfer-db could be built with cabal.++Changes:++* update console-program dependencies+* added NEWS file to keep the history of user visible changes+++transfer-db-0.3.1.0+-------------------++Release date: 2018-03-10++Changes:++* update stack configuration to simplify the build requirements+* add correctivePlan command to generate plans to make corrections to an execution+* add count and drop options to run command (for running plans)+* multithreading parallel proessing of batches+* add dump and restore commands+* a new database agnostic dump library, based on ODBC++transfer-db-0.3.0.0+-------------------++Release date: 2017-10-27++Changes:++* add plan generation functionlity+* modified command line interface to accept commands (run, help)++transfer-db-0.2.0.0+-------------------++Relese date: 2017-09-19++Run the database transfer according to a transfer plan.++Changes:++* read transfer batch from a transfer plan with options to control+ which tables and which data in the tables should be transferred+* commit on every transferred table, so a batch could be restarted++transfer-db-0.1.0.0+-------------------++Release date: 2017-09-15++First stable release of transfer-db application and library. It transfers all+the table in the database schema with no options to control what to be trnasferred.++
README.md view
@@ -7,3 +7,60 @@ application is easily portable to other SQL/CLI implementations. It includes a database agnostic dump file format library.++Current release is 0.3.1.1.++Release Notes+-------------++The dump and restore are not finalized yet, so they don't work with the current version.+++SYNOPSYS+--------++To use transfer-db you either right your own transfer plan by hand+or use transfer-db to generate a transfer plan for you. You can +edit the generated tranfer plan.++A transfer plan is an [yaml](http://yaml.org) file specifying the+database connection parameters and what tables and data should be+transferred.++Gnereating a plan:++ transfer-db makePlan splitByRows <n> -d<SOURCEDB_ODBC_DATASOURCE> -u<SOURCEDB_USER_NAME> -p<SOURCEDB_PASSWORD> -s<SOURCEDB_SCHEMA> -D<DESTDB_ODBC_DATASOURCE> -U<DESTDB_USER_NAME> -P<DESTDB_PASSWORD> -S<DESTDB_SCHEMA> > my-plan.yaml+ +This command will generate on the standard output a plan for transferring the database SOURCEDB +DESTDB in batches of `<n>` rows (the splitByRows sub-command). In case of failure, you could+restart the process by skipping the completed batches.++After that, simply run:++ transfer-db run my-plan.yaml > therun-0001.log++This command will run the plan in the `my-plan.yaml` file and will capture the `stdout` of+`transfer-db` command into the file `therun-0001.log`. This file logs the transferred tables+and batches and could be used to generate a corrective plan in case something went wrong. On+console (`stderr`) there will be printed detailed information about running the plan.++For more advanced options, on controlling the number of threads or of what batches+should be run, type:++ transfer-db help++To generate a corrective plan, you need the original plan and the `stdout` log generated by+the `run` command:++ transfer-db correctivePlan my-plan.yaml -sfix-001.sql -pnew-plan.yaml -itherun-001.log+ +This command will read the original plan from `my-plan.yaml` file, the log of the previous+run from `therun-001.log` file and will generate 2 files:++* `fix-001.sql` that you will need to run to against the destination database in order+ to remove partial batches.+* `new-plan.yaml` that you should use with for a new transfer run, to transfer the remaining+ records.+ ++
transfer-db.cabal view
@@ -1,5 +1,5 @@ name: transfer-db-version: 0.3.1.0+version: 0.3.1.1 -- synopsis: description: Simple SQL/CLI application that transfers data between 2 databases. homepage: http://hub.darcs.net/mihaigiurgeanu/transfer-db@@ -12,6 +12,7 @@ build-type: Simple cabal-version: >=1.10 extra-source-files: README.md,+ NEWS, ChangeLog library@@ -48,7 +49,7 @@ transformers >= 0.5 && < 0.6, yaml >= 0.8.22 && < 0.9, aeson >= 1.0,- console-program >= 0.4.2.0 && < 0.5,+ console-program >= 0.4.2.2 && < 0.5, bytestring >= 0.10.8 && < 0.11, logging >= 3.0 && < 4, time >= 1.8 && < 1.9,