Git SVN: Difference between revisions

From miki
Jump to navigation Jump to search
 
(28 intermediate revisions by 2 users not shown)
Line 13: Line 13:
** [https://git.wiki.kernel.org/images-git/7/78/Git-svn-cheatsheet.pdf git-svn cheatsheet]
** [https://git.wiki.kernel.org/images-git/7/78/Git-svn-cheatsheet.pdf git-svn cheatsheet]
** [https://git.wiki.kernel.org/index.php/GitTips#git_svn git-svn tips]
** [https://git.wiki.kernel.org/index.php/GitTips#git_svn git-svn tips]
* [http://viget.com/extend/effectively-using-git-with-subversion Effectively Using Git with Subversion]


Alternatives to git svn:
* [http://www.subgit.com SubGit]


== Configuration ==
== Configuration ==
My default <tt>.git/config</tt> file on Linux:
Default <tt>.git/config</tt>:
<source lang="bash">
<source lang="bash">
[user]
[user]
Line 21: Line 26:
email = ...
email = ...
[core]
[core]
repositoryformatversion = 0
repositoryformatversion = 0
filemode = false # 'false' STRONGLY advised for cygwin & SVN
filemode = true
bare = false
bare = false
logallrefupdates = true
logallrefupdates = true
ignorecase = true
ignorecase = true # Automatically set by git-svn
autocrlf = false
autocrlf = false # 'false' STRONGLY advised for Linux
whitespace = -blank-at-eol, -space-before-tab,-blank-at-eof
whitespace = -blank-at-eol, -space-before-tab,-blank-at-eof
[apply]
[apply]
ignorewhitespace = change
ignorewhitespace = change
[clean]
requireForce = false # To allow clean -dx w/o force
[svn]
pushmergeinfo = true # So that merge commits are marked as merge (experimental)
[svn-remote "svn"]
[svn-remote "svn"]
url = ...
url = ...
fetch = ...
fetch = ...
branches = ...
branches = ...
tags = ...
tags = ...
</source>
</source>
Optionally the following settings might be added:

<source lang=bash>
My default <tt>.git/config</tt> file on Cygwin:
<source lang="bash">
[user]
name = ...
email = ...
[core]
[core]
bigFileThreshold = 256m # To fix memory overflow issue on compress (mainly cygwin)
repositoryformatversion = 0
filemode = false
bare = false
logallrefupdates = true
ignorecase = true
autocrlf = false
whitespace = -blank-at-eol, -space-before-tab,-blank-at-eof
[apply]
ignorewhitespace = change
[svn-remote "svn"]
url = https://www.collabnet.nxp.com/svn/atop
fetch = trunk:refs/remotes/trunk
branches = branches/*:refs/remotes/*
tags = tags/*:refs/remotes/tags/*
</source>
</source>


== Workflow ==
== Workflow ==
=== Init / Clone ===
Get a clone of subversion repository with:
Get a clone of subversion repository with:
<source lang="bash">
<source lang="bash">
Line 70: Line 63:
</source>
</source>


Create <tt>.gitignore</tt> from SVN ''ignore'' meta-data:
<source lang=bash>
git-svn show-ignore > .gitignore
</source>

=== Fetch / Pull ===
Fetch all changes from SVN repo:
Fetch all changes from SVN repo:
<source lang="bash">
<source lang="bash">
Line 87: Line 86:
:* Use the usual <code>git rebase --abort</code> (to bail out), <code>git rebase --continue</code> (solved conflict), <code>git rebase --skip</code> (no-op patch)
:* Use the usual <code>git rebase --abort</code> (to bail out), <code>git rebase --continue</code> (solved conflict), <code>git rebase --skip</code> (no-op patch)


=== dcommit ===
Send local changes to SVN repo:
Send local changes to SVN repo:
<source lang="bash">
<source lang="bash">
Line 93: Line 93:
</source>
</source>


=== Branch ===
Create a branch using <code>git svn branch</code>. For safety, '''always''' use option <code>-n</code> first (dry-run):
<source lang="bash">
git svn branch -n -m 'Some tricky expirements' mip_tricky # DRY-RUN
git svn branch -m 'Some tricky expirements' mip_tricky # ... if previous command ok
</source>

In case of non-standard project layout with several branch directory specification, we need to use option <code>-d</code> to tell git-svn where to create the branch:
<source lang="bash">
git config --get-all svn-remote.svn.branches
# branches/*:refs/remotes/* <-- we want to create our branch here
# Release_Branches/Dev/*:refs/remotes/Release_Branches/Dev/*
# Release_Branches/Indus/*:refs/remotes/Release_Branches/Indus/*

git svn branch -n -d branches -m 'Some tricky expirements' mip_tricky # DRY-RUN
git svn branch -d branches -m 'Some tricky expirements' mip_tricky # ... if everything's ok
</source>

'''Carefull!''' &mdash; If current commit is not in svn, git svn looks at the nearest svn revision that can be used as parent. This is particularly useful when branching off an existing svn branch or tags. However this '''fails''' if the svn tags have been changed into git tags or if they have been deleted:
<source lang="bash">
git co tags/release_2.3
git svn branch -n -m 'branching off 2.3' mip_tricky # DRY-RUN
# Copying https://..../svn/myproj/tags/release_2.3 at r1478 to .... https://..../branches/mip_tricky

git br -r -D tags/release_2.3 # Let's see what happen if we delete our remote reference
git svn branch -n -m 'branching off 2.3' mip_tricky # DRY-RUN
# Copying https://..../svn/myproj/trunk at r497 to .... https://..../branches/mip_tricky
</source>
After deleting our remote reference, git svn does not use revision 'r1478'' as starting point on tags release_2.3, but the revision ''r497'' from the trunk.

=== Merging ===
With the current ''git svn'' implementation (1.7.5.1), there is '''no way''' to dcommit a merge commit and preserve the merge parents in svn. Although the references to the parents are kept in the git svn repo that pushed the merge, these references will not be seen by other git svn client (the information is simply not there in svn). This means that all revisions created by svn only have a single parent, and merge history is lost.

A way to circumvent this is to immediately create after the merge a revision that will update the <tt>svn:mergeinfo</tt> properties. For this we will need to have a working copy of the svn repositories.

Let's assume we have
- an up-to-date svn repo, working copy of <tt>trunk/</tt>
- a git svn work tree of the same repo, with master branch tracking <tt>trunk/</tt>, and a branch <tt>topicB</tt>, tracking <tt>branches/topicB/</tt>

(See [http://stackoverflow.com/questions/190431/is-git-svn-dcommit-after-merging-in-git-dangerous] for original sequence)

<ol>
<li> First go in the svn repo, and get the latest properties:</li>
<source lang=bash>
svn update
svn propget svn:mergeinfo
</source>
Say you get these info:
<source lang=text>
/branches/topicA:126,183-211
</source>
<li> Now go to git svn repo, dcommit everything, '''don't merge yet''', and get the list of revision we'll need to add after the merge</li>
<source lang=bash>
# Assuming we are on topicB already
git stash # Save your local modifications if needed

# Update and sync master
git checkout master
git svn rebase # We assume we have no changes on master

# Update/sync topicB
git co topicB
git rebase master # topicB is now at the tip of master

# Merge
git checkout master
git merge --no-ff work # Create merge commit

# Check everything
git log --graph --oneline --decorate

git commit --amend # Add a sensible message
git svn dcommit # Push the commit to svn
</source>
<li> Now at this stage, git svn as created new revisions in svn. Some in the branch topicB, and one in the trunk. We need to get the list of revisions that were merged into trunk:</li>
<source lang=bash>
git log master..topicB | egrep -o "@[0-9]+ "
</source>
Say we get:
<source lang=text>
@350
@349
...
@331
@330
</source>
<li> Assume that all revisions are contiguous, in svn notation we have to add the revisions <code>330-350</code> to the mergeinfo property. The new property is then:</li>
<source lang=text>
/branches/topicA:126,183-211
/branches/topicB:330-350
</source>
:'''{{red|Attention!}}''' It is extremelly important to include '''ALL''' revisions in the branch. If one revision is missing, git svn will consider the merge as a cherry-pick and will refuse to mark the revision as a merge.
<li> Update trunk to add the new mergeinfo</li>
<source lang=bash>
cd <svnrepo>
svn update # Get the new revision
svn pg svn:mergeinfo . # Check again the mergeinfo
svn pe svn:mergeinfo . # Add the new merge info as given above
svn commit # Create a new revision with the mergeinfo set
</source>
<li> Now check that everything is fine in git svn:</li>
<source lang=bash>
rm -rf .git/svn/.caches
git svn reset -r 330 -p # Refetch our branch completely
git svn fetch
git svn log --graph --oneline --decorate
</source>
</ol>

'''ATTENTION POINTS''':
* The manual method above is much faster and safer than using the traditional <code>svn merge --record-only ^/branches/topicB</code>:
** First we don't fetch or send useless files over the network as <code>svn merge</code> does.
** Second, in case of special branch structure, like a branch that started off another branch which was not fully merged, etc, svn will fail to set the right revisions number in the mergeinfo, and git-svn will see the merge as a cherry-pick (this is because the last un-commited revision on the 1st branch will be reported for that branch only, and not for the new branch).

==== Using <tt>git svn dcommit --merginfo</tt> ====
Since version '''1.7.8''', we can give the ''mergeinfo'' directly at the ''dcommit'' command.

Say we have branch master, tracking svn ''trunk'', and a branch ''topic'', tracking svn branch ''myfeature'', with revisions on the branch in the range '''1500-1550'''.
<source lang=text>
*-----*-----*-----*-----*-----*-----*-- . . . --*-----* master (-> trunk)
\
\
*-----*-----*-----*-----*-- . . . --*-----* topic (-> myfeature)
r1500 r1503 r1510 ... ... ... r1550
===== =====
</source>

We can then simply merge ''myfeature'' onto ''trunk'' with:
<source lang=bash>
git co master # Check out branch we are merging to
git svn rebase # ... and make sure we are up-to-date
git merge topic # Tell Git to do the merge for us!
</source>
* We can extract the branch name and revision number from the topic branch:
<source lang=bash>
SVNURL=$(git config --get svn-remote.svn.url)
(git log HEAD^..topic | perl -lne "print for m|^ +git-svn-id: *\Q$SVNURL\E(.*@[0-9]+)|g" | sed -r "s/@/:/"; git svn propget svn:mergeinfo) | sort
</source>
* Let's assume we get the following list:
<source lang=text>
/tags/mainline/1.2.5-1.3:560
/tags/mainline/1.2.5-1.3:554
/relbranch/mainline/1.2.x-1.x:550
/relbranch/mainline/1.2.x-1.x:549
/relbranch/mainline/1.2.x-1.x:548
/relbranch/mainline/1.2.x-1.x:547
/relbranch/mainline/1.2.x-1.x:546
/relbranch/mainline/1.2.x-1.x:544
/relbranch/internal/1.2.x-1.x:524
/relbranch/internal/1.2.x-1.x:523
/relbranch/internal/1.2.x-1.x:521
...
...
/relbranch/internal/1.2.x-1.x:507
/relbranch/internal/1.2.x-1.x:488
/relbranch/internal/1.2.x-1.x:487
/vendorbranch/company:1-8
</source>
:We will then compress these into (using an editor):
<source lang=text>
/relbranch/internal/1.2.x-1.x:487-488,507-521,523-524
/relbranch/mainline/1.2.x-1.x:544,546-550
/tags/mainline/1.2.5-1.3:554,560
/vendorbranch/company:1-8
</source>
* Then we can push the merge into svn with:
<source lang=bash>
git svn dcommit --dry-run --mergeinfo="$(cat)" # Paste mergeinfo then Ctrl-D
git svn dcommit --mergeinfo="$(cat)" # Idem, for real now...
</source>

;Old solution:
<source lang=bash>
git svn dcommit --mergeinfo=$'/branches/myfeature:1500-1550\n/branches/prev/merge/info:10-30'
# Now push it to SVN - by giving the mergeinfo, the revision will appear as
# ... a real merge in SVN / other GIT SVN client
# !!! old mergeinfo are ***OVERWRITTEN***, not appended !!!
</source>

* Another solution is to use configuration property "svn.pushmergeinfo", but apparently it only works in specific cases.
* {{red|'''CAUTION'''}} &mdash; '''DON'T FORGET''' to recover first the previous merge info, and append new one.
* {{red|'''CAUTION'''}} &mdash; '''ADD ALL BRANCHES''' on the history list that are to be merged. Indeed some branches might have been created long after the fork. To be sure, include all branches that covers the merge path.

=== Miscellaneous ===
Annotate changes in a file:
Annotate changes in a file:
<source lang="bash">
<source lang="bash">
Line 99: Line 283:
git svn blame fullpath # Only accept full path! (from project rootdir)
git svn blame fullpath # Only accept full path! (from project rootdir)
git blame file
git blame file
</source>

Create git <tt>.gitignore</tt> from svn:
<source lang="bash">
# Typically in project root:
git-svn show-ignore > .gitignore
</source>

=== More Workflows ===
* http://ujihisa.blogspot.com/2010/10/git-svn-low-risk-practice.html
* http://stackoverflow.com/questions/190431/is-git-svn-dcommit-after-merging-in-git-dangerous

== Config with non-standard layout ==

In case the SVN repo does not follow the standard directory layout (i.e. <tt>trunk/</tt>, <tt>branches/</tt> and <tt>tags</tt>), we can change the default config. Here an example of config:

<source lang="text">
[user]
name = beq06659
email = michael.peeters@nxp.com
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
autocrlf = false
whitespace = -blank-at-eol,-space-before-tab,-blank-at-eof
[apply]
ignorewhitespace = change
[svn-remote "svn"]
url = https://www.collabnet.nxp.com/svn/atop
fetch = trunk:refs/remotes/trunk
branches = branches/*:refs/remotes/*
# branches = Release_Branches/Customer/Audi/*:refs/remotes/Release_Branches/Customer/Audi/*
# branches = Release_Branches/Customer/S1NN/*:refs/remotes/Release_Branches/Customer/S1NN/*
# branches = Release_Branches/Dev/*:refs/remotes/Release_Branches/Dev/*
# branches = Release_Branches/Indus/*:refs/remotes/Release_Branches/Indus/*
# tags = tags/*:refs/remotes/tags/*
tags = tags/Customer/*:refs/remotes/tags/Customer/*
tags = tags/Industrial/Telebox/*:refs/remotes/tags/Industrial/Telebox/*
tags = tags/Industrial/*:refs/remotes/tags/Industrial/*
tags = tags/Intermediate/*:refs/remotes/tags/Intermediate/*
tags = tags/Mainline/*:refs/remotes/tags/Mainline/*
</source>

;Do's and Don't's
* We see that there are overlapping tag specification (<tt>tags/Industrial/Telebox/*</tt> and <tt>tags/Industrial/*</tt>). This is because some tags are created in the parent directory, and some more are to be found in the <tt>Telebox</tt> sub-directory. This will create a conflict though because git will create a reference named <tt>tags/Industrial/Telebox</tt>, and later will try to create a reference <tt>tags/Industrial/Telebox/...</tt> that fails because The first reference prevents creating the sub-directory. To solve, simply remove the reference and restart:
<source lang="bash">
git branch -r -D tags/Industrial/Telebox
git svn fetch
</source>
* Git svn falls back in ''stupid mode'' (i.e. fetching the full history of each branch, even if given revision was already fetched in the past) if it is initialized without any option. So '''always init / clone the repo with option <tt>-s</tt> (standard layout)'''. If the layout is non-standard, use <code>git svn init</tt> and edit the config.
* Never change the default location of branch. Never use a spec like <code> branches = branches/*:refs/remotes/branches/*</code>. When trying with such a spec, git svn was fetching and fetching again the same revisions (from <tt>r1</tt>!).

== Tips ==
=== Remove orphaned svn branch ===
:See [http://stackoverflow.com/questions/5914681/git-svn-bulk-removing-orphaned-remote-branches] (and simple fix at [http://stackoverflow.com/questions/1839606/delete-a-svn-branch-via-git])

=== Convert tag branch to real git tags ===
:See [http://gitready.com/advanced/2009/02/16/convert-git-svn-tag-branches-to-real-tags.html]. My script:
<source lang="bash">
#! /bin/bash

# Get .git directory
GITDIR=$(git rev-parse --git-dir)
if [ "$GITDIR" = "" ]; then
echo 'FAILED! Not in a git repository...'
exit 1
fi

# Backup current tag references
echo "Saving svn tags reference to $GITDIR/svn_tags_backup..."
SVNTAGBACKUP="$GITDIR/svn_tags_backup"
if [ -a "$SVNTAGBACKUP" ]; then
read -i N -p "$SVNTAGBACKUP already exists. Overwrite? [y/N] "
if [ "$REPLY" != "y" -a "$REPLY" != "Y" ]; then
exit 1
fi
fi

# Convert the tags
git-for-each-ref refs/remotes/tags > "$SVNTAGBACKUP"
< "$SVNTAGBACKUP" cut -d / -f 4- |
while read REF
do
# Adding tag comment does not work... go figure why
# COMMENT=$(git log --pretty=tformat:"%s" -1 "refs/remotes/tags/$REF")
# git tag -a "svn/$REF" -m "$COMMENT" "refs/remotes/tags/$REF"
git tag "svn/$REF" "refs/remotes/tags/$REF"
git branch -r -d tags/$REF
done
</source>

=== Force git svn to re-fetch a branch ===
:See [https://gist.github.com/372883]. Basically it amounts to <code>rm .git/svn/git-svn/mynewbranch -Rf && git svn fetch</code>.

A cleaner way might actually be to use <code>git svn reset</code>:
<source lang=bash>
rm -rf .git/svn/.caches # Clear the cache
git svn reset -r 300 # set 300 as last fetched revision, use -p to fetch 300 also.
git svn fetch
</source>

If given revision must be fetched again, use option <code>-p</code>.

My first reset did fail with some error similar to the one we get when the cache is corrupted. Clearing and resetting/fetching again did the trick though.

=== Convert git ''diffs'' into svn ''diffs'' ===
:See [http://mojodna.net/2009/02/24/my-work-git-workflow.html]

=== Edit git-svn-id after cherry-pick ===

This will remove the ''git-svn-id'' tag, but add the original author name:
<source lang=bash>
git filter-branch -f --msg-filter 'sed -r -e "s/^[ \t]*git-svn-id: (.*@[0-9]+) .*/Original SVN commit by $GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL> \1/"' cp747..HEAD
</source>

This will remove the ''git-svn-id'' tag but the reference to original revision number (add ''Cherry-picked from:'' prefix):
<source lang=bash>
git filter-branch -f --msg-filter "sed -r -e 's/^[ \t]*git-svn-id:(.*)[ \t]+.*/Cherry-picked from: \1/g'" <your svn branch>..HEAD
</source>
</source>


== Troubleshooting ==
== Troubleshooting ==
=== Bugs ===
=== Byte order not compatible ===
* '''Byte order not compatible''' &mdash; Recently got the following error on cygwin:
* Recently got the following error on cygwin:
<source lang="text">
<source lang="text">
Byte order is not compatible at ../../lib/Storable.pm (autosplit into ../../lib/auto/Storable/_retrieve.al) line 380, at /usr/share/perl/5.10.1/Memoize/Storable.pm line 21
Byte order is not compatible at ../../lib/Storable.pm (autosplit into ../../lib/auto/Storable/_retrieve.al) line 380, at /usr/share/perl/5.10.1/Memoize/Storable.pm line 21
Line 112: Line 416:
<source lang="bash">
<source lang="bash">
rm -r .git/svn/caches
rm -r .git/svn/caches
</source>

=== Git svn fetches several times the same revision ===
If git svn does not detect that a branch is a branch starting from the trunk, it will fetch the complete branch history from the beginning, possibly fetching a revision that was fetched already. This is because svn is very flexible on branch, and git svn has to take the pessimistic choice to make sure no data is lost.
* [http://osdir.com/ml/git/2009-07/msg01665.html git svn fetches the same revision multiple times for non-trunk branches]
* [http://stackoverflow.com/questions/1140428/git-svn-fetch-retrieves-the-same-subversion-revision-multiple-times-for-branches git svn fetch retrieves the same Subversion revision multiple times for branches]<br/>This post seems to tell us that we should change the refspec of remote svn branches (i.e. leave them in <tt>refs/remotes/*</tt>)

A possible solution is to limit the range of revisions (see [http://stackoverflow.com/questions/2029520/can-git-svn-be-used-on-large-branched-repositories/2378750#2378750]):
<source lang="bash">
git svn log --all -1 | \
sed -n '2s/r\\([0-9]*\\).*/\\1/p' | \
xargs --replace=from git svn fetch -r from:HEAD
</source>
Another solution would be to fetch by range (0-100, 100-200, etc).

=== Lots of @rev ===
Likely to occur when tags have been moved around (see [http://stackoverflow.com/questions/5961848/git-svn-creates-a-lot-of-branches-appended-with-rev]).

A short script to remove them:
<source lang="bash">
git tag -l svn/* | grep @ |
while read tag; do
DUPL=$(git tag --contains "$tag" | wc -l)
if [ $DUPL != 0 ]; then
git tag -d "$tag"
fi
done
</source>
</source>

Latest revision as of 09:44, 6 September 2013

This page is dedicated to Subversion integration in Git

References


Alternatives to git svn:

Configuration

Default .git/config:

[user]
    name = ...
    email = ...
[core]
    repositoryformatversion = 0
    filemode = false                       # 'false' STRONGLY advised for cygwin & SVN
    bare = false
    logallrefupdates = true
    ignorecase = true                      # Automatically set by git-svn
    autocrlf = false                       # 'false' STRONGLY advised for Linux
    whitespace = -blank-at-eol, -space-before-tab,-blank-at-eof
[apply]
    ignorewhitespace = change
[clean]
    requireForce = false                   # To allow clean -dx w/o force
[svn]
    pushmergeinfo = true                   # So that merge commits are marked as merge (experimental)
[svn-remote "svn"]
    url = ...
    fetch = ...
    branches = ...
    tags = ...

Optionally the following settings might be added:

[core]
    bigFileThreshold = 256m                # To fix memory overflow issue on compress (mainly cygwin)

Workflow

Init / Clone

Get a clone of subversion repository with:

# git: git clone => 
# svn: svn checkout =>
git svn clone https://www.mysvn.org/svn/project -T trunk -b branches -t tags --username='USER'
git svn clone https://www.mysvn.org/svn/project --stdlayout --username='USER'                   # Identical
# Clean / compress git DB if necessary...
git gc

Create .gitignore from SVN ignore meta-data:

git-svn show-ignore > .gitignore

Fetch / Pull

Fetch all changes from SVN repo:

# git: git fetch =>
git svn fetch

Fetch and merge all changes from SVN repo:

# git: git pull =>
# svn: svn update =>
git svn rebase
git svn rebase -l                   # Only rebase against last fetched commit from upstream SVN
In case of conflicts (see [1]):
  • You are on a (no-branch) branch (with git status showing a .dotest file you can ignore)
  • Use the usual git rebase --abort (to bail out), git rebase --continue (solved conflict), git rebase --skip (no-op patch)

dcommit

Send local changes to SVN repo:

# git: git push =>
git svn dcommit

Branch

Create a branch using git svn branch. For safety, always use option -n first (dry-run):

git svn branch -n -m 'Some tricky expirements' mip_tricky                  # DRY-RUN
git svn branch -m 'Some tricky expirements' mip_tricky                     # ... if previous command ok

In case of non-standard project layout with several branch directory specification, we need to use option -d to tell git-svn where to create the branch:

git config --get-all svn-remote.svn.branches
# branches/*:refs/remotes/*                                              <-- we want to create our branch here
# Release_Branches/Dev/*:refs/remotes/Release_Branches/Dev/*
# Release_Branches/Indus/*:refs/remotes/Release_Branches/Indus/*

git svn branch -n -d branches -m 'Some tricky expirements' mip_tricky      # DRY-RUN
git svn branch -d branches -m 'Some tricky expirements' mip_tricky         # ... if everything's ok

Carefull! — If current commit is not in svn, git svn looks at the nearest svn revision that can be used as parent. This is particularly useful when branching off an existing svn branch or tags. However this fails if the svn tags have been changed into git tags or if they have been deleted:

git co tags/release_2.3
git svn branch -n -m 'branching off 2.3' mip_tricky                        # DRY-RUN
# Copying https://..../svn/myproj/tags/release_2.3 at r1478 to .... https://..../branches/mip_tricky

git br -r -D tags/release_2.3                                              # Let's see what happen if we delete our remote reference
git svn branch -n -m 'branching off 2.3' mip_tricky                        # DRY-RUN
# Copying https://..../svn/myproj/trunk at r497 to .... https://..../branches/mip_tricky

After deleting our remote reference, git svn does not use revision 'r1478 as starting point on tags release_2.3, but the revision r497 from the trunk.

Merging

With the current git svn implementation (1.7.5.1), there is no way to dcommit a merge commit and preserve the merge parents in svn. Although the references to the parents are kept in the git svn repo that pushed the merge, these references will not be seen by other git svn client (the information is simply not there in svn). This means that all revisions created by svn only have a single parent, and merge history is lost.

A way to circumvent this is to immediately create after the merge a revision that will update the svn:mergeinfo properties. For this we will need to have a working copy of the svn repositories.

Let's assume we have - an up-to-date svn repo, working copy of trunk/ - a git svn work tree of the same repo, with master branch tracking trunk/, and a branch topicB, tracking branches/topicB/

(See [2] for original sequence)

  1. First go in the svn repo, and get the latest properties:
  2. svn update
    svn propget svn:mergeinfo
    

    Say you get these info:

    /branches/topicA:126,183-211
    
  3. Now go to git svn repo, dcommit everything, don't merge yet, and get the list of revision we'll need to add after the merge
  4. # Assuming we are on topicB already
    git stash                         # Save your local modifications if needed
    
    # Update and sync master
    git checkout master
    git svn rebase                    # We assume we have no changes on master
    
    # Update/sync topicB
    git co topicB
    git rebase master                 # topicB is now at the tip of master
    
    # Merge
    git checkout master
    git merge --no-ff work            # Create merge commit
    
    # Check everything
    git log --graph --oneline --decorate
    
    git commit --amend                # Add a sensible message
    git svn dcommit                   # Push the commit to svn
    
  5. Now at this stage, git svn as created new revisions in svn. Some in the branch topicB, and one in the trunk. We need to get the list of revisions that were merged into trunk:
  6. git log master..topicB | egrep -o "@[0-9]+ "
    

    Say we get:

    @350
    @349
    ...
    @331
    @330
    
  7. Assume that all revisions are contiguous, in svn notation we have to add the revisions 330-350 to the mergeinfo property. The new property is then:
  8. /branches/topicA:126,183-211
    /branches/topicB:330-350
    
    Attention! It is extremelly important to include ALL revisions in the branch. If one revision is missing, git svn will consider the merge as a cherry-pick and will refuse to mark the revision as a merge.
  9. Update trunk to add the new mergeinfo
  10. cd <svnrepo>
    svn update                       # Get the new revision
    svn pg svn:mergeinfo .           # Check again the mergeinfo
    svn pe svn:mergeinfo .           # Add the new merge info as given above
    svn commit                       # Create a new revision with the mergeinfo set
    
  11. Now check that everything is fine in git svn:
  12. rm -rf .git/svn/.caches
    git svn reset -r 330 -p          # Refetch our branch completely
    git svn fetch
    git svn log --graph --oneline --decorate
    

ATTENTION POINTS:

  • The manual method above is much faster and safer than using the traditional svn merge --record-only ^/branches/topicB:
    • First we don't fetch or send useless files over the network as svn merge does.
    • Second, in case of special branch structure, like a branch that started off another branch which was not fully merged, etc, svn will fail to set the right revisions number in the mergeinfo, and git-svn will see the merge as a cherry-pick (this is because the last un-commited revision on the 1st branch will be reported for that branch only, and not for the new branch).

Using git svn dcommit --merginfo

Since version 1.7.8, we can give the mergeinfo directly at the dcommit command.

Say we have branch master, tracking svn trunk, and a branch topic, tracking svn branch myfeature, with revisions on the branch in the range 1500-1550.

*-----*-----*-----*-----*-----*-----*-- . . . --*-----*  master (-> trunk)
       \
        \
         *-----*-----*-----*-----*-- . . . --*-----*     topic  (-> myfeature)
       r1500 r1503  r1510 ...   ...         ...  r1550
       =====                                     =====

We can then simply merge myfeature onto trunk with:

git co master                                                # Check out branch we are merging to
git svn rebase                                               # ... and make sure we are up-to-date
git merge topic                                              # Tell Git to do the merge for us!
  • We can extract the branch name and revision number from the topic branch:
SVNURL=$(git config --get svn-remote.svn.url)
(git log HEAD^..topic | perl -lne "print for m|^ +git-svn-id: *\Q$SVNURL\E(.*@[0-9]+)|g" | sed -r "s/@/:/"; git svn propget svn:mergeinfo) | sort
  • Let's assume we get the following list:
/tags/mainline/1.2.5-1.3:560
/tags/mainline/1.2.5-1.3:554
/relbranch/mainline/1.2.x-1.x:550
/relbranch/mainline/1.2.x-1.x:549
/relbranch/mainline/1.2.x-1.x:548
/relbranch/mainline/1.2.x-1.x:547
/relbranch/mainline/1.2.x-1.x:546
/relbranch/mainline/1.2.x-1.x:544
/relbranch/internal/1.2.x-1.x:524
/relbranch/internal/1.2.x-1.x:523
/relbranch/internal/1.2.x-1.x:521
...
...
/relbranch/internal/1.2.x-1.x:507
/relbranch/internal/1.2.x-1.x:488
/relbranch/internal/1.2.x-1.x:487
/vendorbranch/company:1-8
We will then compress these into (using an editor):
/relbranch/internal/1.2.x-1.x:487-488,507-521,523-524
/relbranch/mainline/1.2.x-1.x:544,546-550
/tags/mainline/1.2.5-1.3:554,560
/vendorbranch/company:1-8
  • Then we can push the merge into svn with:
git svn dcommit --dry-run --mergeinfo="$(cat)"            # Paste mergeinfo then Ctrl-D
git svn dcommit --mergeinfo="$(cat)"                      # Idem, for real now...
Old solution
git svn dcommit --mergeinfo=$'/branches/myfeature:1500-1550\n/branches/prev/merge/info:10-30'
                                                             # Now push it to SVN - by giving the mergeinfo, the revision will appear as 
                                                             # ... a real merge in SVN / other GIT SVN client
                                                             # !!! old mergeinfo are ***OVERWRITTEN***, not appended !!!
  • Another solution is to use configuration property "svn.pushmergeinfo", but apparently it only works in specific cases.
  • CAUTIONDON'T FORGET to recover first the previous merge info, and append new one.
  • CAUTIONADD ALL BRANCHES on the history list that are to be merged. Indeed some branches might have been created long after the fork. To be sure, include all branches that covers the merge path.

Miscellaneous

Annotate changes in a file:

# git: git blame =>
# svn: svn annotate/blame =>
git svn blame fullpath              # Only accept full path! (from project rootdir)
git blame file

Create git .gitignore from svn:

# Typically in project root:
git-svn show-ignore > .gitignore

More Workflows

Config with non-standard layout

In case the SVN repo does not follow the standard directory layout (i.e. trunk/, branches/ and tags), we can change the default config. Here an example of config:

[user]
      name = beq06659
      email = michael.peeters@nxp.com
[core]
      repositoryformatversion = 0
      filemode = true
      bare = false
      logallrefupdates = true
      autocrlf = false
      whitespace = -blank-at-eol,-space-before-tab,-blank-at-eof
[apply]
      ignorewhitespace = change
[svn-remote "svn"]
      url = https://www.collabnet.nxp.com/svn/atop
      fetch = trunk:refs/remotes/trunk
      branches = branches/*:refs/remotes/*
#     branches = Release_Branches/Customer/Audi/*:refs/remotes/Release_Branches/Customer/Audi/*
#     branches = Release_Branches/Customer/S1NN/*:refs/remotes/Release_Branches/Customer/S1NN/*
#     branches = Release_Branches/Dev/*:refs/remotes/Release_Branches/Dev/*
#     branches = Release_Branches/Indus/*:refs/remotes/Release_Branches/Indus/*
#     tags = tags/*:refs/remotes/tags/*
      tags = tags/Customer/*:refs/remotes/tags/Customer/*
      tags = tags/Industrial/Telebox/*:refs/remotes/tags/Industrial/Telebox/*
      tags = tags/Industrial/*:refs/remotes/tags/Industrial/*
      tags = tags/Intermediate/*:refs/remotes/tags/Intermediate/*
      tags = tags/Mainline/*:refs/remotes/tags/Mainline/*
Do's and Don't's
  • We see that there are overlapping tag specification (tags/Industrial/Telebox/* and tags/Industrial/*). This is because some tags are created in the parent directory, and some more are to be found in the Telebox sub-directory. This will create a conflict though because git will create a reference named tags/Industrial/Telebox, and later will try to create a reference tags/Industrial/Telebox/... that fails because The first reference prevents creating the sub-directory. To solve, simply remove the reference and restart:
git branch -r -D tags/Industrial/Telebox
git svn fetch
  • Git svn falls back in stupid mode (i.e. fetching the full history of each branch, even if given revision was already fetched in the past) if it is initialized without any option. So always init / clone the repo with option -s (standard layout). If the layout is non-standard, use git svn init and edit the config.
  • Never change the default location of branch. Never use a spec like branches = branches/*:refs/remotes/branches/*. When trying with such a spec, git svn was fetching and fetching again the same revisions (from r1!).

Tips

Remove orphaned svn branch

See [3] (and simple fix at [4])

Convert tag branch to real git tags

See [5]. My script:
#! /bin/bash

# Get .git directory
GITDIR=$(git rev-parse --git-dir)
if [ "$GITDIR" = "" ]; then
    echo 'FAILED! Not in a git repository...'
    exit 1
fi

# Backup current tag references
echo "Saving svn tags reference to $GITDIR/svn_tags_backup..."
SVNTAGBACKUP="$GITDIR/svn_tags_backup"
if [ -a "$SVNTAGBACKUP" ]; then
    read -i N -p "$SVNTAGBACKUP already exists. Overwrite? [y/N] "
    if [ "$REPLY" != "y" -a "$REPLY" != "Y" ]; then
        exit 1
    fi
fi

# Convert the tags
git-for-each-ref refs/remotes/tags > "$SVNTAGBACKUP"
< "$SVNTAGBACKUP" cut -d / -f 4- |
while read REF
do
    # Adding tag comment does not work... go figure why
    # COMMENT=$(git log --pretty=tformat:"%s" -1 "refs/remotes/tags/$REF")
    # git tag -a "svn/$REF" -m "$COMMENT" "refs/remotes/tags/$REF"
    git tag "svn/$REF" "refs/remotes/tags/$REF"
    git branch -r -d tags/$REF
done

Force git svn to re-fetch a branch

See [6]. Basically it amounts to rm .git/svn/git-svn/mynewbranch -Rf && git svn fetch.

A cleaner way might actually be to use git svn reset:

rm -rf .git/svn/.caches        # Clear the cache
git svn reset -r 300           # set 300 as last fetched revision, use -p to fetch 300 also.
git svn fetch

If given revision must be fetched again, use option -p.

My first reset did fail with some error similar to the one we get when the cache is corrupted. Clearing and resetting/fetching again did the trick though.

Convert git diffs into svn diffs

See [7]

Edit git-svn-id after cherry-pick

This will remove the git-svn-id tag, but add the original author name:

git filter-branch -f --msg-filter 'sed -r -e "s/^[ \t]*git-svn-id: (.*@[0-9]+) .*/Original SVN commit by $GIT_AUTHOR_NAME <$GIT_AUTHOR_EMAIL> \1/"' cp747..HEAD

This will remove the git-svn-id tag but the reference to original revision number (add Cherry-picked from: prefix):

git filter-branch -f --msg-filter "sed -r -e 's/^[ \t]*git-svn-id:(.*)[ \t]+.*/Cherry-picked from: \1/g'" <your svn branch>..HEAD

Troubleshooting

Byte order not compatible

  • Recently got the following error on cygwin:
Byte order is not compatible at ../../lib/Storable.pm (autosplit into ../../lib/auto/Storable/_retrieve.al) line 380, at /usr/share/perl/5.10.1/Memoize/Storable.pm line 21
[...] 
Could not unmemoize function `lookup_svn_merge', because it was not memoized to begin with at /usr/lib/git-core/git-svn line 3197
Basically git svn cannot read the content of its cache, and crashes. This can happen after an update of Perl, or when using a copy of a repository that has been created on another machine (typ. with different endianness settings) (see [8] and [9]).
The best fix is to simply delete the git svn cache:
rm -r .git/svn/caches

Git svn fetches several times the same revision

If git svn does not detect that a branch is a branch starting from the trunk, it will fetch the complete branch history from the beginning, possibly fetching a revision that was fetched already. This is because svn is very flexible on branch, and git svn has to take the pessimistic choice to make sure no data is lost.

A possible solution is to limit the range of revisions (see [10]):

git svn log --all -1 | \
  sed -n '2s/r\\([0-9]*\\).*/\\1/p' | \
  xargs --replace=from git svn fetch -r from:HEAD

Another solution would be to fetch by range (0-100, 100-200, etc).

Lots of @rev

Likely to occur when tags have been moved around (see [11]).

A short script to remove them:

git tag -l svn/* | grep @ | 
while read tag; do 
    DUPL=$(git tag --contains "$tag" | wc -l)
    if [ $DUPL != 0 ]; then 
        git tag -d "$tag"
    fi
done