Unison: Difference between revisions

From miki
Jump to navigation Jump to search
 
(3 intermediate revisions by the same user not shown)
Line 16: Line 16:


=== Alternatives to Unison ===
=== Alternatives to Unison ===
* [https://syncthing.net/ SyncThing] (link fails in Opera)
* [https://syncthing.net/ SyncThing] ([https://github.com/syncthing/syncthing github], lot of praise on [https://news.ycombinator.com/item?id=27149002 HN])
* [https://www.getsync.com/ Bit-torrent Sync] (see [https://thetinhat.com/tutorials/alternatives/btsync.html this tutorial])
* [https://www.getsync.com/ Bit-torrent Sync] (see [https://thetinhat.com/tutorials/alternatives/btsync.html this tutorial])
* [https://joeyh.name/blog/entry/gitless/ git as an alternative to unison]
* [https://github.com/magic-wormhole/magic-wormhole magic-wormhole] (file transfer)
:* Alternative to [https://github.com/schollz/croc croc].


=== Limitations ===
=== Limitations ===
Line 36: Line 39:


=== Unison vs rsync ===
=== Unison vs rsync ===
By default, Unison uses its own implementation of rsync algorithms for diffs-only transfer. But for whole-file copy of new file, this implementation is not very optimized. Unison can be configured to address these shortcomings.
By default, Unison uses its own implementation of rsync algorithms for diffs-only transfer. But for whole-file copy of new file, this implementation is not very optimized, and will be slower than regular rsync for big files. Unison can be configured to address these shortcomings. For instance []:

copythreshold = 1024

This will default to using rsync for whole file copy of files bigger than 1MB. For smaller file, Unison will use its own implementation.


== Basic usage ==
== Basic usage ==

Latest revision as of 10:55, 25 May 2021

References


Unison also has built-in help:

unison -help               # Brief help
unison -doc topics         # Get list of details documentation
unison -doc all            # Get the whole manual

Alternatives to Unison

  • Alternative to croc.

Limitations

  • (from the manual) Unison does not understand hardlinks. So copying a repo with hardlinks will just create a new repo where all hard-links are unique (unlinked) copies.
mkdir repo1 repo2
echo foo > repo1/foo
ln repo1/foo repo1/bar
unison -batch -ignorearchives repo1 repo2
find -type f -printf '%n %p\n'
# 2 ./repo1/foo
# 2 ./repo1/bar
# 1 ./repo2/foo
# 1 ./repo2/bar
  • Do NOT use Unison to sync git repositories. Indeed, if both repositories changes, Unison conflict resolution may corrupt them. However do not ignore them either because ignored files/folder may be deleted by Unison if the parent directory is renamed, hence leading to information loss [1]. Instead, move git repositories outside unison replica roots, and use git to sync these repos. If these repos must appear in the unison replica anyhow, then consider using symlinks.

Unison vs rsync

By default, Unison uses its own implementation of rsync algorithms for diffs-only transfer. But for whole-file copy of new file, this implementation is not very optimized, and will be slower than regular rsync for big files. Unison can be configured to address these shortcomings. For instance []:

copythreshold = 1024

This will default to using rsync for whole file copy of files bigger than 1MB. For smaller file, Unison will use its own implementation.

Basic usage

  • Synchronize two directories
unison <dir-a> <dir-b>
  • Selecting files to synchronize
    • Synchronize home directory, using Ignore facility to skip some files.
    • Create a subdirectory called shared, and synchronize that directory only.
    • (Linux only) Create a subdirectory called shared, with symbolic links to other local directories, and synchronize that directory only using follow (making symlink transparent).
    • Synchronize home directory as root, using -path switch on the command line:
      unison /home/username ssh://remotehost//home/username -path shared
      

      The -path option can be used as many times as needed, to synchronize several files or subdirectories:

      unison /home/username ssh://remotehost//home/username \
          -path shared \
          -path pub \
          -path .netscape/bookmarks.html
      

      These -path arguments can also be put in your preference file.

Configuration tips

  • On Samba / CIFS file system, ignore permissions:
perms = 0
In some cases, you also need to use the preferences dontchmod.
  • When running unison with ssh, uses servercmd if unison is not found in the path:
servercmd = /usr/bin/unison

Basic Profile

Here we assume that all profiles will include the default one. If not, a more complex solution is to create a file .unison/common-prefs, that would be included by all profiles, including the default one.

Here a basic setup to backup pen usb stick on Cygwin:

Compile from sources

Compile on powerpc/arm

References:

  • Instructions from Tom Booschaert ([2]).

These are instructions to compile unison from sources on NAS devices with powerpc/arm cpu, and using ipkg package management.

# install OCaml, Gnu C Compiler en Make:
ipkg install ocaml
ipkg install gcc
ipkg install make

# get Unison source and unpack:
wget http://www.seas.upenn.edu/~bcpierce/unison//download/releases/stable/unison-2.32.52.tar.gz
# Other versions:
# wget http://www.seas.upenn.edu/~bcpierce/unison//download/releases/unison-2.27.57/unison-2.27.57.tar.gz
# wget http://www.seas.upenn.edu/~bcpierce/unison//download/releases/unison-2.40.102/unison-2.40.102.tar.gz
tar -xzf unison-2.32.52.tar.gz

# use make to compile:
cd unison-2.32.52
make NATIVE=false UISTYLE=text

# copy executable to bin folder so it can be executed anywhere:
cp unison /opt/bin/
Compilation error
  • Error /bin/sh: etags: not found can be freely ignored.
  • When building with ocaml 3.12, we get the following error:
ocamlc -I lwt -I ubase -custom -g -c /root/build/tmp/unison-2.27.57/update.ml
File "/root/build/tmp/unison-2.27.57/update.ml", line 1, characters 0-1:
Error: The implementation /root/build/tmp/unison-2.27.57/update.ml
       does not match the interface update.cmi:
       Modules do not match:
This is a known bug. Patch file update.mli as follows (see [3]):
sed -ri '/^module NameMap : Map/s/Map.S/MyMap.S/' update.mli
  • On ARM, we had the following error:
+ gcc -o 'unison'   '-Llwt' '-Lubase' '-L/opt/lib/ocaml'  '/tmp/camlprimed354e.c' '-lcamlstr' '-lunix' '-lutil' 'osxsupport.o' 'pty.o' '-lcamlrun' 
-I'/opt/lib/ocaml' -lm  -ldl  -lpthread
/opt/lib/gcc/arm-none-linux-gnueabi/4.2.3/../../../../arm-none-linux-gnueabi/bin/ld: Warning: /lib/libc.so.6: Unknown EABI object attribute 44
/opt/lib/gcc/arm-none-linux-gnueabi/4.2.3/../../../../arm-none-linux-gnueabi/lib/libdl.so: undefined reference to `_dl_tls_get_addr_soft@GLIBC_PRIVATE'
To fix it we temporarily link to a different libdl.so library [4]:
mv /opt/arm-none-linux-gnueabi/lib/libdl.so /opt/arm-none-linux-gnueabi/lib/libdl.so.old
ln -s /lib/libdl.so.2 /opt/arm-none-linux-gnueabi/lib/libdl.so
# Compile Unison with the new library:
make NATIVE=false UISTYLE=text
# Restore the library link
mv /opt/arm-none-linux-gnueabi/lib/libdl.so.old /opt/arm-none-linux-gnueabi/lib/libdl.so
test Unison
unison -version
# unison version 2.27.57
unison -selftest
# Contacting server...
# Connected [//Mnemosyne//root/test-a.tmp -> //Mnemosyne//root/test-b.tmp]
# Running internal tests...
# backups 1 (local)...
# backups 2...
# backups 2a...
# backups 3...
# backups 4...
# backups 5 (directories)...
# backups 6 (backup prefix/suffix)...
# links 1 (directories and links)...
# links 2 (symlink to nowhere)...
# Success :-)