Git: Difference between revisions

Revision as of 01:32, 4 April 2021

References

Help

git help branch              # LONG help on 'git branch'
man git-branch               # ... idem
man git branch               # ... idem
git branch -h                # SHORT help on 'git branch'

Links

Git cheat sheet

Git Home
Git Tutorial
Git on Ubuntu
Pro Git
Git, from the bottom up
Linux Greatest Invention
Tech Talk: Linux Torvalds on git
Git cheat sheet
Remove Sensitive Data from Git
Example on how to use git-filter-branch to delete files and propagates this upstream. See also BFG below.
Git from the inside out.
Git Internals (book,PDF) (locate peepcode-git_internals.pdf).
WYAG - Write Yourself A Git

Detailed explanation on how to write a Git compatible tool, not so hard according to author.

Things to look at

Why arent you using git flow?
Quick overview of git-flow, a small to implement this git branching model
Visualize Git with SeeGit
Visual aid showing off the structure of a git repository in real time while issuing commands against the repository.

Git vs ...

Mercurial vs Git — It's all in the branches
Comparing Hg with Git, by a long-time advocate of Git ;-) (see also [1])
Mercurial, Subversion, and Wesley Snipes
Including several posts on various git-related topics.
More: [2], [3].

Git Usage Statistics

According to recent statistics, Git is growing faster than other DVCS system, and would be 10 times more used than e.g. Mercurial:

GNOME Survey, Debian popularity, Stack Overflow, Programmers StackExchange (indicating 1 order of magnitude difference between Git and Hg in Ohloh and Eclipse survey).

Git on Windows

http://superuser.com/questions/138669/why-is-tortoise-git-changing-my-file-permissions
Might require git config core.filemode false
Using cygwin git instead of msysgit with TortoiseGit

Tools

git-annex
diff-highlight (see also tip further below)
new diff-so-fancy
BFG Repo-Cleaner

Tutorials

Learn Git Branching interactively

Game-like, animated, etc.

Git SVN

Subversion integration

See Git SVN page

Introduction

Git Features:

Reliability
Performance
Distributed

Distributed

Originally from BitKeeper. Other distributed SCM is Mercurial.

No single repository. Everybody always has his own copy of the repository. The repository content is pulled from other people's repository.
No politics, no commit access control. All work is always done locally, so there is no need to define such politics.

Reliability

Every change, file, directory, etc. is cryptographically hashed (sha1sum).

Easy corruption detection. Any tampering to a file or directory content (either malicious or because of hardware failure) is immediately detected.
Easy distribution. Moreover because the repository is distributed all over the place, it is very easy to repair a given repository. You only need to drop all broken objects, and get all missing objects from a remote copy.

Performance

Very fast commit. Local repository

Terminology and Concepts

gitrevisions

A commit is a snapshot of your working tree at some point in time. A revision in git denotes either a given commit, or all commit that can be reached from that commit. There are different ways to name a commit (see also man gitrevisions):

branchname — a branch name is an alias for most recent commit on that branch
tagname — similar to a branch alias, but that does not change in time
HEAD — currently checked out commit
c82a22c — the SHA-1 hash id of the commit (can be truncated as long as it remains unique)
name^ — the parent of commit name
name^^ — the grand-parent of commit name (and so on)
name^2 — the 2nd parent of commit name (and so on)
name~10 — the 10th ancestor of commit name (same as name^^^^^^^^^^)
name:path — reference a specific file/directory in a given commit
name^{tree} — reference the tree held by a commit
name1..name2 — a commit range, i.e. all commits reachable from name2 back to, but no including, name1 (if either name is omitted, use HEAD instead)
name1...name2 — refers to all commits referenced by name1 or name2, but not by both. For git diff, refers to all commits between name2 and the common ancestor of name1 and name2.
master.. — to review changes made to the current branch
..master — after a fetch, to review all changes occured since last rebase or merge
--since="2 weeks ago" — all commits since a certain date
--until=”1 week ago” — all commits up to a certain date
--grep=pattern — all commits whose commit message matches the regular expression pattern.
--committer=pattern — all commits whose committer matches the pattern
--author=pattern — all commits whose author matches the pattern
--no-merges — all commits in a range that have only one pattern (i.e. ignore all merge commits)

detached head: When HEAD is no longer a reference to anything (like ref: refs/heads/branch), but instead contains the actual hash of a commit.

git checkout -b newbranch           # To attach HEAD back on a new branch...

hunk: individual change within a file (basically a file diff output is made of a sequence of one or more hunks).

Install

From packages

Install the following essential packages:

git-core — the main program
git-gui — a gui front-end

Optionally install also:

git-doc — documentation
gitweb — Web interface
ViewGit — Another web interface
gitosis — Project management:
tig — a text-mode repository browser interface to git and color pager.

tig                                # launch browser
git show | tig                     # Use as pager. Colorize output of git-show

gitview — Git Repository browser
gitg — a Git repository browser targeting Gtk+ / GNOME

ppa:pasgui/ppa

sudo apt-add-repository ppa:pasgui/ppa

or alternatively, create a file /etc/apt/sources.list.d/pasgui-ppa-lucid.list (change lucid as necessary):

deb http://ppa.launchpad.net/pasgui/ppa/ubuntu lucid main 
deb-src http://ppa.launchpad.net/pasgui/ppa/ubuntu lucid main

Then add the apt key:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F599ACE3

qgit — A graphical interface to git repositories using QT

From sources

Get the sources with

git clone git://github.com/gitster/git.git          # Use proxygit if behind a proxy
cd git
git checkout v1.7.8                                 # To use given version

Some dependencies must be installed:

sudo apt-get install autoconf zlib1g-dev libcurl4-openssl-dev expat asciidoc     # More packages might be needed
sudo apt-get install docbook2x                                                   # For info pages

For cygwin:

Install the packages above (or similar)
install package docbook-xml45
if needed (see INSTALL):

xmlcatalog --noout --add rewriteURI http://docbook.sourceforge.net/release/xsl/current /usr/share/sgml/docbook/xsl-stylesheets /etc/xml/catalog
xmlcatalog --noout --add rewriteURI http://www.oasis-open.org/docbook/xml/4.5/xsl/current /usr/share/sgml/docbook/xml-dtd-4.5 /etc/xml/catalog

Add temporarily an executable script ~/bin/xmlto:

#! /bin/bash
/usr/bin/xmlto --skip-validation "$@"

To build and install (again, see INSTALL):

make configure                                           # See also INSTALL
./configure --prefix=/usr/local                          # In // of existing package installation. /usr/local has precedence
make all doc info                                        # 'info' optional, more dependencies
sudo make install install-doc install-html install-info  # 'install-info' optional

In MSYS2

MSYS2 provides its won package for git. However it is slower that Git-for-Windows. So the simplest is to add the path of Git-for_Windows into MSYS2:

echo 'export PATH="$PATH:/c/Program Files/git/mingw64/bin"' > /etc/profile.d/git-for-windows.sh

Configuration

References:

Git Community Boot - Customizing Git
Git handy feedback on command-line

General

Global per-user configuration settings are stored in file ~/.gitconfig

Add color to git output for all commmands:

git config --global color.ui true

Define author/email

git config --global user.name "Your Name"
git config --global user.email you@example.com

Add some frequently used aliases:

git config --global alias.st    'status'
git config --global alias.ci    'commit'
git config --global alias.co    'checkout'
git config --global alias.br    'branch'
git config --global alias.ls    'ls-files'
git config --global alias.last  'log -1 HEAD'
# h like history
git config --global alias.h     'log --oneline --graph --decorate -45'
git config --global alias.hb    'log --oneline --graph --decorate -45 --branches'
git config --global alias.ha    'log --oneline --graph --decorate -45 --all'
# l like log ;-)
git config --global alias.l     'log --pretty=tformat:\"%C(yellow)%h %Cblue%<(15,trunc)%an %Cgreen%<(12,trunc)%cr%C(auto)%d %Creset%s\" --graph -45'
git config --global alias.lb    'log --pretty=tformat:\"%C(yellow)%h %Cblue%<(15,trunc)%an %Cgreen%<(12,trunc)%cr%C(auto)%d %Creset%s\" --graph -45 --branches'
git config --global alias.la    'log --pretty=tformat:\"%C(yellow)%h %Cblue%<(15,trunc)%an %Cgreen%<(12,trunc)%cr%C(auto)%d %Creset%s\" --graph -45 --all'
git config --global alias.j     'log --pretty=tformat:\"%C(yellow)%h %Cblue%<(15,trunc)%an %Cgreen%ci%C(auto)%d %Creset%s\" --graph -45'
git config --global alias.jb    'log --pretty=tformat:\"%C(yellow)%h %Cblue%<(15,trunc)%an %Cgreen%ci%C(auto)%d %Creset%s\" --graph -45 --branches'
git config --global alias.ja    'log --pretty=tformat:\"%C(yellow)%h %Cblue%<(15,trunc)%an %Cgreen%ci%C(auto)%d %Creset%s\" --graph -45 --all'
git config --global alias.dc    'diff --cached'
git config --global alias.wdiff 'diff --color-words'
git config --global alias.wshow 'show --color-words'

More aliases for submodules:

git config --global alias.push   'push --recurse-submodules=on-demand'
git config --global alias.sst    '!git status && git submodule foreach git status'
git config --global alias.sdiff  '!git diff $@ && git submodule foreach git diff $@'
git config --global alias.sclean '!git clean $@ && git submodule foreach git clean $@'
git config --global alias.sreset '!git reset $@ && git submodule foreach git reset $@'
git config --global alias.sh     '!git h -1 && git submodule foreach git h -1'
git config --global alias.su     'submodule'
git config --global alias.suu    'submodule update'

Setup global .gitattributes file (for better tokenization with git diff --color-words):

git config --global core.attributesfile '~/.gitattributes'

Content of ~/.gitattributes:

*.c    diff=cpp
*.cpp  diff=cpp

Some handy scripts:

git-wtf displays the state of your repository in a readable and easy-to-scan format

Global ~/.gitignore_global file (see [4]):

# Global .gitignore file that applies to all git repositories
#
# Add this file to git global config with
#
#   git config --global core.excludesfile ~/.gitignore_global

# Vim temp files
.*.swp

Add this file with:

git config --global core.excludesfile ~/.gitignore_global

Filter script to prevent running git clean in HOME directory:

#! /bin/bash

if [ -a /usr/local/bin/git ]; then
  ORIG_GIT=/usr/local/bin/git
else
  ORIG_GIT=/usr/bin/git
fi

for c; do
    if [ "$c" = "clean" ]; then
      GIT_HOME="$HOME/.git"
      GIT_DIR="$($ORIG_GIT rev-parse --git-dir 2>/dev/null)"
      GIT_HOME_STAT=$(stat -c %i "$GIT_HOME")
      GIT_DIR_STAT=$(stat -c %i "$GIT_DIR")
      if [ "$GIT_HOME_STAT" = "$GIT_DIR_STAT" -o "$GIT_HOME" = "$GIT_DIR" ]; then
        echo "Can't invoke 'git clean' in HOME git repository!"
        exit 1
      fi
      break                                   # Break on 'clean'
    fi
    if [[ $c != -* || $c == -- ]]; then
        break                                 # ... or on first non-option arg or --
    fi
done

"$ORIG_GIT" "$@"

Disable pager for git log:

git config --global pager.log false           # Can be enabled back with 'git -l log'

Enable auto-complete for git aliases

alias g=git
alias gg='git -p'
complete -F _git g                          # Enable auto-complete for these aliases too
complete -F _git gg

Setting the tab width in git diff outputs [5]

git config --global core.pager 'less -x1,5'

Use the credential helper to cache username / password when connecting to a remote via HTTPS [6]:

git config --global credential.helper cache                     # Setup cache with default timeout (900s)
git config --global credential.helper 'cache --timeout=3600'    # To change the default timeout

Some command configuration:

git config --global fetch.recurseSubmodules on-demand           # TODO: Maybe better to set to true
git config --global fetch.prune true

gitk and tcl/tk

To solve the issue of ugly fonts (not anti-aliased besides other uglyness), install tk8.5 and force alternatives for wish (see [7]):

sudo apt-get install tk8.5
sudo update-alternatives --config wish
# select wish8.5

An alternative however is to use gitg.

To use Gnome GTK native look, install tileqt as tcl/qt theme (see [8], instructions from [9] ):

# Install tileqt build deps
sudo apt-get install tcl8.5-dev tk8.5-dev libqt4-dev checkinstall


# Fetch, configure, build tileqt
cd ~/tmp
proxygit clone git://tktable.git.sourceforge.net/gitroot/tktable/tile-qt
cd tile-qt
# On UBUNTU LUCID:
./configure --with-tcl=/usr/lib/tcl8.5/ --with-tk=/usr/lib/tk8.5/
# On UBUNTU PRECISE (multi-arch), pick the correct arch:
./configure --with-tcl=/usr/lib/tcl8.5/ --with-tk=/usr/lib/tk8.5/ --with-qt-include=/usr/include/qt4 --with-qt-lib=/usr/lib/i386-linux-gnu
./configure --with-tcl=/usr/lib/tcl8.5/ --with-tk=/usr/lib/tk8.5/ --with-qt-include=/usr/include/qt4 --with-qt-lib=/usr/lib/x86_64-linux-gnu
make
sudo checkinstall                                             # Answer questions until files are copied

# Set 'tileqt' as default theme, and update system (w/o reboot)
echo '*TkTheme: tileqt' >> ~/.Xresources
xrdb -merge ~/Xresources

More info/themes:

Issues:

Does not work on Ubuntu Precise (at least under VB). We get the message
X error of failed request: BadDrawable (invalid Pixmap or Window parameter)

Hook - prevent non-fast-forwarded update

References:

http://git-scm.com/book/en/Customizing-Git-An-Example-Git-Enforced-Policy

Save this file as .git/hooks/update to install the hook. Edit configuration settings hooks.allownonffupdatemaster to allow / forbid (default) non-fast-forward update.

#!/bin/sh
#
# A hook to prevent non fast-forward update of 'master'.
#
# Config
# ------
# hooks.allownonffupdatemaster
#   This boolean sets whether non-fast-forwarded update on master are allowed.
#   By default, they won't be to prevent history loss.

# --- Command line
refname="$1"
oldrev="$2"
newrev="$3"

# --- Safety check
if [ -z "$GIT_DIR" ]; then
	echo "Don't run this script from the command line." >&2
	echo " (if you want, you could supply GIT_DIR then run" >&2
	echo "  $0 <ref> <oldrev> <newrev>)" >&2
	exit 1
fi

if [ -z "$refname" -o -z "$oldrev" -o -z "$newrev" ]; then
	echo "Usage: $0 <ref> <oldrev> <newrev>" >&2
	exit 1
fi

# --- Config
allownonffupdatemaster=$(git config --bool hooks.allownonffupdatemaster)

# echo "update hook: refname '$refname' oldrev '$oldrev' newrev '$newrev' allownonffupdatemaster '$allownonffupdatemaster'"

case "$refname" in
	refs/heads/master)
		if [ "$allownonffupdatemaster" != "true" ]; then
		    # update is not a fast-forward if there are commits that we can access from oldrev, but not from newrev
		    is_non_ff=$(git rev-list $newrev..$oldrev|wc -l)
		    if [ $is_non_ff -ne 0 ]; then
		        echo "*** [hook] Non-fast-forward updates on master are not allowed, even if forced."
		        exit 1
		    fi
		fi
		;;
	*)
		# Anything else allowed
		exit 0
		;;
esac

# --- Finished
exit 0

Install / Configure git-daemon

References:

git-daemon is the daemon that enables access to git repositories through the git: protocol (port 9418). It is easily installed thanks to package git-daemon-sysvinit:

sudo apt-get install git-daemon-sysvinit
sudo vi /etc/default/git-daemon             # Change user to 'git'
sudo /etc/init.d/git-daemon start           # Start daemon

By default you cannot access any git repository through git: protocol. You must explicitly list which repository are accessible:

cd ~git/git/project.git
touch git-daemon-export-ok                                       # This allow git-daemon to export this project
sudo ln -sf ~git/git/project.git /var/cache/git/project.git      # In addition, project must be listed in /var/cache/git

Note that base address for git-daemon is /var/cache. Hence, given example above, one would clone it with git clone git://server/git/project.git.

Write access are also refused by default. To allow them, you must edit the .git/config file:

cd ~git/git/project.git
git config --add daemon.receivepack true    # no hyphen in receivepack!

CR/LF conversion

To disable CR/LF completely, across all copies of the repositories, create a file .gitattributes in the root of the repository with the following as content [10]:

* -text

This should no other side-effect than disabling CRLF/LF conversion, and requires no further configuration, including on new/old clones.

Install Git Server with SSH / git-shell

Here we access a central git server with ssh, and limit access by using the git shell git-shell.

Create the new user git, using git-shell as shell.

useradd --create-home --skel /dev/null --home-dir /home/git --shell /usr/bin/git-shell git
chmod 750 /home/git

Note: We choose /home/git as root for our repositories because it makes obvious that this folder must be backed up. Another possibility is to use /var/lib/git, the debian default; /var/git, to match var/www used in apache; /srv/git recommended in SELinux and ArchLinux. [11]

Create .ssh/authorized_keys file:

mkdir .ssh; chmod 700 .ssh
touch .ssh/authorized_keys; chmod 600 .ssh/authorized_keys
chown -R git:git ~git

Add user public key to file above, but also make sure to add no-agent-forwarding,no-port-forwarding,no-X11-forwarding before each key to prevent port forwarding, etc. For instance:

no-agent-forwarding,no-port-forwarding,no-X11-forwarding ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCrpFEALD473OqeplM+qyBx+46KMquWK9utwNmvIy3iBatE4S2oFrHVjeQjSNNDK9qaVh2cHPNzzB0UkV9y174ZLn9qeatU032ieKgFfdTdgQCe88BM8eSiVxTkVWe/bawQwn0qqghPtQl7v6/Bof9H9pAGeWLPTfUFj/+CQTf2vevBZF7iuh/RhaokJ75EY29E2lMlAgfIsLv8OVt71LBJNzvXaZIvQgFn03et44UZuZpoqYlZZgAuVgJbHG88QRxcvciBl3H7adkphEkoGUn05JpG2G01Yw2orxpKuMAkCs6t+Z7LjzQhDe+vmKYC4ZUNr5APDwSUxovI3HWJ10Xv mip@home

GitHub repositories

Set a local name and email for GitHub repositories. For instance:

git config user.name "Michaël Peeters"
git config user.email "xeyownt@users.noreply.github.com"

Or use script github-config.sh:

#! /bin/bash

if [ -d .git ]; then
    read -p "Configure current git repo with gitgub user [y/N] ? "
    case $REPLY in
        y|Y|yes|YES)
            set -x
            git config user.name "Michaël Peeters"
            git config user.email "xeyownt@users.noreply.github.com"
            set +x
            echo "Done."
            ;;
        *)
            echo "Aborted."
    esac
else
    echo "Not a git repository"
    exit 1
fi

To correct wrong address:

git filter-branch --commit-filter '
    export GIT_AUTHOR_NAME="Michaël Peeters";
    export GIT_AUTHOR_EMAIL="xeyownt@users.noreply.github.com";
    export GIT_COMMITTER_NAME="Michaël Peeters";
    export GIT_COMMITTER_EMAIL="xeyownt@users.noreply.github.com";
    git commit-tree "$@"' -- basecommit..HEAD

Commands

Here we'll summarize how to use some of the Git commands

Frequently used & common command options

Option	Valid for	Description
`-p, -u, --patch`	`log`	Generate patch
`-b`	`log, diff, show`	Ignore whitespace changes
`--oneline`	`log, show`	Show commit # + description on oneline
`-M`	`log, diff, show`	Find renames
`--summary`	`log, diff, show`	Show summary of changes (file creation, rename...)
`--name-status`	`log, diff, show`	List file name and type of change (Add, Copied, Deleted, Modified, Renamed)

git-add

git-add adds file contents to the index

git add -A                                    # Stage all modified AND new files in current directory and recursively

git-archive

git-archive creates an archive of file from a named tree

# Duplicate current subtree at version commit in ~/tmp/project/sub
# !!! DON'T FORGET *trailing slash* in --prefix
git archive --prefix project/sub/ commit . | tar -x -C ~/tmp   
# Same as above
git archive --prefix project/ commit sub | tar -x -C ~/tmp

git-blame

git blame -w file                                       # Blame file, ignore whitespaces
git blame file | sort | uniq -w 8 | egrep -o "^.{8}"    # Show list of commits that produced file
                                                        # (! actually a subset of git log)

git-branch

git-branch lists, creates, or deletes branches

# CREATE a local branch
git branch newbranch                          # Create a new local branch 'newbranch'
                                              # ... use "git checkout newbranch" to check it out
 
# TRACK a remote branch
git branch --track sf origin/serverfix        # Create a local branch 'sf' that tracks remote branch 'serverfix'
git branch --set-upstream sf origin/serverfix # ... same, but when local branch 'sf' already exists
 
# DELETE a branch
git branch -d sf                              # Delete local branch 'sf'
git branch -d -r origin/serverfix             # Delete remote tracking branch 'serverfix' (see remark below)
git remote prune origin                       # Prune all state remote tracking branch

# MOVE a branch
git branch -f branch commit                   # Move tip of an existing branch to a different commit

# VIEW branches
git branch --all                              # List ALL branches
git branch --contains commit                  # List LOCAL branches that CONTAIN commit
git branch --all --contains commit            # List ALL branches that CONTAIN commit

Note:

You can also track local branch (git branch -t local1 local2), but is it useful?
Deleting remote tracking branch (git branch -d -r) on the local repository only makes sense if the remote branch has been deleted on the remote, or if git-fetch has been configured not to import that branch anymore. So the best is simply to prune remote tracking branches automatically:

git remote prune origin       # Remove all remote tracking branches that no longer exist on the remote (i.e. stale branches)
git fetch --prune             # After fetching, remove all remote tracking branches that no longer exist on the remote

git-checkout

git-checkout checkouts a branch or paths to the working tree.

# CHECKOUT a local branch
git checkout mybranch                         # Checkout an existing branch

# CHECKOUT the N-th last checked-out branch
git checkout @{-2}                            # Checkout the 2nd last branch before current
git checkout -                                # Similar to "@{-1}"

# CREATE & CHECKOUT a local branch
git checkout -b newbranch                     # Create a new local branch 'newbranch' and check it out
 
# TRACK & CHECKOUT a remote branch
git checkout --track origin/serverfix         # Checkout a new local branch 'serverfix' to track remote branch 'serverfix'
                                              #   (remember that this branch is called locally 'origin/serverfix')
git checkout -b sf origin/serverfix           # ... same as above, but the local branch is named 'sf'

# Checkout <paths> from INDEX or COMMIT
git checkout -- <paths...>                    # DISCARD changes
git checkout COMMIT -- <paths...>             # Same as "git reset commit -- path; git checkout -- path"

git-clean

git-clean removes untracked files from the working tree.

git clean -dfx                                # force + delete dirs + delete also ignored files

git-clone

git-clone is mainly used to create a local copy of a remote repository, or to create a bare repository (i.e. one without a working tree) for remote storage:

Clone a remote repository:

git clone git@griffin:repositories/myproject.git  # Clone repository and create working tree in myproject/

Create a bare repository for remote storage:

git clone --bare myproject myproject.git          # Create a bare clone of your repository, if not available yet
scp -r myproject.git/ git@griffin:repositories/   # Copy the repository to server - requires SSH access
rm -rf myproject.git                              # Delete local bare clone

The command git clone /dir/repo/project.git is identical to running the following commands:

git init                                            # Create an empty repo
git remote add -f origin /dir/repo/project.git      # Add a remote repo called 'origin' and fetch
git set-head -a                                     # Set default remote branch for remote 'origin' automatically
git checkout --track origin/master                  # Create a tracking branch 'master', and update working tree

Another equivalent option for last command is git checkout -b origin origin/master (since the start-point is remote, git creates a tracking branch).

Some variants:

To get a local copy of a remote repository, but without changing the working tree (i.e. keeping all local changes), just change the last command to:

git init
git remote add -f origin -m master /dir/repo/project.git
git branch --track master origin/master             # Branch 'master' set up to track remote branch 'master' from 'origin'

To merge remote branch locally, but without creating a tracking branch, change the last command to :

# ...
git merge origin/master                             # Merge

git-commit

git commit -m "commit message"    # Gives immediately the commit message on the command-line
git commit -a                     # Add all changes and commit in one pass
git commit --amend                # Amend tip current branch (message, add some files) - also for merge commits
git commit -C <commit>            # Reuse commit message from <commit>

Add the following line to your file ~/.gitconfig:

git config --global alias.ci 'commit'

Now you can use ci instead of commit:

git ci -m "commit message"

git-diff

git-diff shows changes between commits, commit and working tree, etc

Sample of useful options (some can be combined):

git diff --stat <commit>                # Show changes summary as a graph (with + and -)
git diff -b <commit>                    # Ignore whitespaces
git diff -w <commit>                    # Ignore all whitespaces (remove them completely)
git diff --name-only <commit>           # Only list names of changed files
git diff --relative[=<dir>] <commit>    # Restrict changes to current (or given) directory and show relative pathname
git diff -M --summary <commit>          # Show summary of changes (incl. file creation), find renames

git-fetch

git-fetch downloads objects and refs from another repository.

git fetch -p                      # After fetching, remove any remote-tracking branches which no longer exist on the remote
                                  # (see also git remote prune origin)

git-filter-branch

git-filter-branch rewrites branches.

This command can be used for instance to rewrite / rebase a branch while changing the author/committer name/email (see [12] and [13]):

# Change the author and committer name / email
git filter-branch --commit-filter '
    export GIT_AUTHOR_NAME="...";
    export GIT_AUTHOR_EMAIL="...";
    export GIT_AUTHOR_DATE="...";
    export GIT_COMMITTER_NAME="...";
    export GIT_COMMITTER_EMAIL="...";
    export GIT_COMMITTER_DATE="...";
    git commit-tree "$@"' -- basecommit..HEAD

# Set the committer name to author name, for instance after a rebase
git filter-branch --commit-filter '
    export GIT_COMMITTER_NAME="$GIT_AUTHOR_NAME"; 
    export GIT_COMMITTER_EMAIL="$GIT_AUTHOR_EMAIL"; 
    export GIT_COMMITTER_DATE="$GIT_AUTHOR_DATE"; 
    git commit-tree "$@"' -- basecommit..HEAD

# Change the author/committer name for specific commit (e.g. if wrong email was used):
git filter-branch --commit-filter '
        if [ "$GIT_COMMITTER_NAME" = "<Old Name>" ];
        then
                GIT_COMMITTER_NAME="<New Name>";
                GIT_AUTHOR_NAME="<New Name>";
                GIT_COMMITTER_EMAIL="<New Email>";
                GIT_AUTHOR_EMAIL="<New Email>";
                git commit-tree "$@";
        else
                git commit-tree "$@";
        fi' HEAD

Note that basecommit..HEAD can easily be changed to other commit specification, like --since="1 year ago"

Another example to remove sensitive files from repository (from [14]):

# Delete the file(s)
git clone git@github.com:defunkt/github-gem.git
cd github-gem/
git filter-branch --index-filter 'git rm --cached --ignore-unmatch pattern1 pattern2 ...' HEAD 
# .... Add --tag-name-filter "cat" to keep tags - will overwrite existing tags!

# Push to origin
git push origin master --force

# Force cleanup (a safer method is to clone the repository and keep the clone if everything's fine)
rm -rf .git/refs/original/
git reflog expire --expire=now --all
git gc --prune=now
git gc --aggressive --prune=now

Deleting a directory in all commits

# -- --all apply to all branches
# filter-branch -f to ignore backup in refs/original/
git filter-branch -f --index-filter 'git rm --ignore-unmatch --cached -rfq shared/saves' -- --all

Moving a directory

# Rewrite revision history - Move directory (since we branched off master)
git filter-branch --tree-filter 'mv subsystem/inc inc' master..HEAD

Applying on all branches

git filter-branch ... HEAD will only filter all commits that are reachable from HEAD. To filter all branches, simply replace HEAD by -- --all (see [15]):

git filter-branch --tree-filter 'mv subsystem/inc inc' -- --all

Move a directory and symlink

Same as above but we want to keep a symlink to the new location. It appears we can't move and symlink at the same time because Git complains about following files through symlinks. So we proceed in 2 steps:

git filter-branch --tree-filter "moveit nfc4java bb/firmware_35g/option/jvm mv" nfcbase..HEAD
git filter-branch --tree-filter "moveit nfc4java bb/firmware_35g/option/jvm ln" nfcbase..HEAD

moveit is a script in the path with the following code:

#! /bin/bash

SRC=$1
DST=$2
PHASE=$3             # either 'mv' (1st phase) or 'ln' (2nd phase)

if [ "$PHASE" = "mv" ]; then
    # 1st phase

    mv $SRC/nfc.h $DST/include/nfc.h
    mv $SRC/nfc.c $DST/src/nfc.c
    if [ -a $SRC/fri-lib ]; then 
        rm -rf $DST/nfc-fri
        mv $SRC/fri-lib $DST/fri-lib
    else
        rm -rf $DST/nfc-fri
        mv $SRC/nfc-fri $DST/nfc-fri
    fi
else
    # 2nd phase

    ln -sf ../$DST/include/nfc.h $SRC/nfc.h
    ln -sf ../$DST/src/nfc.c $SRC/nfc.c
    if [ -a $DST/fri-lib ]; then
        mv $DST/fri-lib $DST/nfc-fri
        ln -sf ../$DST/nfc-fri $SRC/fri-lib
    else
        ln -sf ../$DST/nfc-fri $SRC/nfc-fri
    fi
fi

As a result the file / directory nfc.h, nfc.c, fri-lib, nfc-fri are moved from nfc4java/ to bb/firmware_35g/option/jvm, and symlinks are created in the source directory so these files / directories are still accessible.

Adding a CR/PR tag to each commmit

Say we need to add a CR/PR tag to each of our commit (like [CR_1234]), we can do it with:

git filter-branch -f --msg-filter "sed -r '1 s/$/ [CR_1234]/'" master..HEAD

git-grep

git-grep prints lines matching a pattern (see also Git Book)

# Find all occurences of pattern in all files committed since last year
for i in $(git log --oneline --all --graph --since="1 year ago" | egrep -o " [a-h0-9]{7} "); do git grep pattern $i; done

Maybe a better solution from [16]:

git grep <regexp> $(git rev-list --all)

git-log

git-log shows commit logs.

git log                                                # Standard history log
git log -5                                             # Limit to 5 commits
git log -- file                                        # List commits affecting file
git log -p -- file                                     # History log, show patch/diff for file
git log -p -M -- file                                  #  ... idem, but find also renames
git log --summary -M -- file                           # Condensed summary with renames
git log --stat -1                                      # Show diff-stat for last commit
git log -w master.. --stat -- $(find . -name "*.java") # Show all changes to java files since branching off master
git log -p topic..HEAD -- .                            # Show all changes done in current dir that are not in topic branch
git log --all --ancestry-path A..                      # Show all commits that are descendants of A (also with gitk)

Some handy aliases:

[alias]
    # h like history
    h = log --oneline --graph --decorate -45
    ha = log --oneline --graph --decorate -45 --all
    # l like log ;-)
    l = log --pretty=tformat:\"%C(yellow)%h %Cblue%<(15,trunc)%an %Cgreen%<(12,trunc)%cr%C(auto)%d %Creset%s\" --graph -45
    la = log --pretty=tformat:\"%C(yellow)%h %Cblue%<(15,trunc)%an %Cgreen%<(12,trunc)%cr%C(auto)%d %Creset%s\" --graph -45 --all

View history of a single files, across copy and renames:

git l --follow -- file                                  # Show all commits that modified file (incl. renames)
                                                        # (includes commits that deleted lines, unlike blame)
git log -b --follow -p -- file                          # ... same, show patch and ignore blanks

CAUTION! — git log will not show any differences for merge commits! To fix that, use options -m, -c or --cc (see [17]):

git log -p <commit>            # SHOW NOTHING!!!
git log -p -m <commit>         # Show 2 diffs, one for each parent
git log -p -c <commit>         # ... Same as above, but simultaneously
git log -p --cc <commit>       # ... Same as above, but remove hunks that are identical to one of the parent

CAUTION! — git log --oneline --graph output can be misleading! Overlooking the graph may give the impression that a given commit is in some branch where actually it is in another branch. Look at the asterisks (*) carefully!. Example:

* | 2dbfa34 frq05215 4 weeks ago Integration implementation...
* |   cd1d5a7 beq03416 4 weeks ago Merged head of trunk into...
|\ \  
* | | 84fb520 frq05215 4 weeks ago Integration implementation...  # This commit is not in same branch as 0eef5c7!
| | | * 0eef5c7 beq03416 4 weeks ago relaxed the rate adaptation...
| | |/  
| | * b998ebd nxp12661 4 weeks ago [PT_TLMT2086] Changed bool_t type into J9bool_t...

git-ls-tree

git ls-tree -r COMMIT [path...]
# 100644 blob 33161fa40c2b89322852e80517fa4610165e2874	some/file
# 160000 commit efa26f89222e5422038147af51107fbb0ea8c17d	submoduleA
# 160000 commit 25adb6ed7dc53e9b60450ed16b2ff7772feef02f	submoduleB
# 100644 blob 6fad0f8a30ff202c7566925cd96c31edcf96597c	file

This allows to see the content of git repository:

files appears as blob objects.
submodules appears as commit objects.

This is very convenient to see which commit a git submodule is pointing to:

git ls-tree -r COMMIT | grep commit # may fail if a filename contains "commit"

git-pull

git-pull fetches from and merges with another repository or a local branch.

git pull -p                   # After fetching, remove any remote-tracking branches which no longer exist on the remote
                              # (see also git remote prune)

git-push

git push updates remote refs along with associated objects. In layman english, git push basically pushes changes to the remote repository, possibly creating, updating, deleting branches, objects or references.

# PUBLISH a local branch (for TRACKing)
# git push [remotename] [localbranch]:[remotebranch]
git push                                      # Push current branch to remote
git push origin                               # Push all matching branches to remote
git push origin serverfix                     # Push a local branch 'serverfix' to remote (create it if necessary)
git push -u origin serverfix                  # ... same but also set upstream reference for TRACKING
git push -u origin serverfix:serverfix        # ... same as above
git push -u origin serverfix:coolfix          # ... same but call the branch 'coolfix' on the remote

# DELETE a branch
git push origin :serverfix                    # Delete remote branch 'serverfix' on 'origin'
                                              # (basically this means push nothing to remote branch 'serverfix')

git-rebase

git-rebase forward-port local commits to the updated upstream head.

git rebase master                                  # Rebase current branch to master branch
git rebase master topic                            # Rebase branch topic to branch master. Same as 
                                                   #   git checkout topic
                                                   #   git rebase master
git rebase master topic --onto newbase             # Same as above, but instead write the commits at dest
git rebase --committer-date-is-author-date master  # ... keep original (author) date for new commits

git-reflog

git reflog manages reflog information. This is very handy to repair mistakes, or to recover lost commits (e.g. after modifying the head of a branch that was the only reference to a given commit)

git reflog

git-remote

Add another remote to an existing repository. Using git remote set-head and git branch one call also set up automatically origin/HEAD and set tracking branches. For instance the following is equivalent to what's done by git clone:

git remote add -f origin git@griffin:repositories/my_project.git  # Add remote repository and fetch automatically
git remote set-head -a origin                                     # Set origin/HEAD automatically - see man git-remote, set-head
git branch --set-upstream master origin                           # Set master to track head (here origin/master) branch from origin

git-remote set-head: Sets / deletes the default branch for a remote. For remote origin, this creates the reference refs/remotes/origin/HEAD with content ref: refs/remotes/origin/master if default branch is master.
Use set-head to follow the changes in another branch than the default one:

git remote set-head origin -a                       # Set default remote branch for remote 'origin' automatically
git diff origin                                     # -> Will show difference with origin/master (if 'master' is the default)
git remote set-head origin exoticbranch             # ... or set it to a different branch (here 'exoticbranch')
git diff origin                                     # -> Now will show diff with origin/exoticbranch

git-reset

git reset has two purposes:

commit paths... → Index
Copy paths... from HEAD (or given commit) to index (working tree left unchanged), or
branch head → commit
Move current branch head to HEAD (or given commit), and optionally update index and/or working tree.

mode	index	working tree	preserve local changes
`--soft`			Yes
`--mixed` (default)	reset		Yes
`--hard`	reset	reset	No
`--merge`	reset	keep if idx = commit, else reset if idx = tree, else abort	No (fully staged changes are lost)
`--keep`	reset	keep if HEAD = commit, else reset if HEAD = tree, else abort	Yes

Note:

git reset copies the old head to ORIG_HEAD.
Use git checkout <commit> -- <paths...> to copy paths... from a given commit to the index and working tree in one go.
Caution! — Don't move the head of a branch if changes have been already published, or this will create many merge conflicts.
Caution! — git reset --hard is a dangerous command. Prefer one of its alternatives, like:

Commit any changes first

git commit -a -m "snapshot WIP"
git reset --hard~3

Stash the changes

git stash
git reset --hard HEAD~3
# ...
git reset --hard HEAD@{1} # or ORIG_HEAD
git stash apply

Idem & don't change master too early

git stash
git checkout -b new-branch HEAD~3
...
git branch -D master
git branch -m new-branch master

Some use cases (see man git-reset for details):

Undo a commit and redo

git commit ...
git reset --soft HEAD^
edit
git commit -a -c ORIG_HEAD # or -C

Undo commits permanently

git commit ...
git reset --hard HEAD~3

Undo a commit, making it a topic branch

git branch topic/wip
git reset --hard HEAD~3 # or --mixed
git checkout topic/wip  # or -m topic/wip

Undo a merge or pull

git pull                # conflicts
git reset --hard
git pull . topic/branch # no conflict
git reset --hard ORIG_HEAD

Undo a merge or pull inside a dirty work tree

git pull
git reset --merge ORIG_HEAD

Interrupted workflow

git checkout feature
#work work work
git commit -a -m "snapshot WIP"
git checkout master
#fix fix fix
git commit
git checkout feature
git reset HEAD^     # or --soft

Reset a single file in the index

git add foo.c
git reset -- foo.c

git-show

git-show shows various types of objects.

git show --stat <commit>         # Show commit; for merge, show *combined* commit
git show -b -R <commit>          # Show reverse patch, ignoring blanks (handy for reversing commit by hand)
git show <treeish>:<file>        # Show file at given commit (see also 'git checkout')
git show -s <commit>             # Show commit, but suppress diff output (like 'git log -1')

git-submodule

See Submodules section.

git-verifypack

View the content of .pack files:

# List all objects in a .pack file
git verify-pack -v ./objects/pack/pack-8b93785651b5f573728ddb84c8857c831c9c4535.idx

Submodules

References:

http://git-scm.com/book/en/v2/Git-Tools-Submodules

Summary of command 'git submodule'

git submodule add
git submodule status
git submodule init
git submodule deinit
git submodule update [--init] [--recursive]
git submodule summary [--cached] [--files]           # See also 'git diff --submodule=log'
git submodule foreach
git submodule sync

Clone a project with submodules

git clone <project_url>        # First clone parent project. submodule folders will be present but empty
git submodule init             # Init local configuration file
git submodule update           # Fetch all data from submodule project, and check out appropriate commit

git clone <project_url>
git submodule update --init    # Idem, one command less

git clone --recursive <project_url>  # Same as above, all in one step

Add a submodule into an existing project

git submodule add <public_url> [submodulename]

To use another, private, URL to pull from locally, use

git config submodule.<submodulename>.url <private_url>

Diff with pretty submodules info

git diff --submodule                     # View commit changes in submodules

git config --global diff.submodule log   # To avoid typing --submodule everytime

Updating a local submodule to get upstream changes

cd <submodulename>
git fetch
git merge origin/master        # or git merge --ff-only

or simpler

git submodule update --remote [submodulename]   # Update one or all submodules

By default, commands above will update the checkout to the 'master' branch of submodule directory. To track a different branch do

git config -f .submodules submodule.<submodulename>.branch <branchname>    # for everyone checking out parent project
git config submodule.<submodulename>.branch <branchname>                   # locally only

Git status can also show a summary of changes (new commit)

git config status.submodulesummary 1                                       # Tell git-status to show summary of changes
git status

Update submodule and merge / rebase

git submodule update --remote --merge
git submodule update --remote --rebase

Publishing submodules changes

Make sure to push new commits in submodules first before updating the parent.

git push --recurse-submodules=check                    # Check that submodule remote contains the necessary commit
git push --recurse-submodules=ondemand                 # Push missing submodules commit to remote if necessary

See also option 'submodule.<name>.fetchRecurseSubmodules' to have either options above by default

Update submodule url

Edit .gitmodules, then run

git submodule sync
git submodule update

Tips

Run a command in all submodules

git submodule foreach 'git stash'
git submodule foreach 'git checkout -b featureA'
git diff; git submodule foreach 'git diff'

Useful aliases

git config alias.sdiff '!' "git diff && git submodule foreach 'git diff'"
git config alias.spush 'push --recurse-submodules=on-demand"
git config alias.supdate 'submodule update --remote --merge'

Pits

Committing changes in a submodule in detached head state, then updating the module to a different commit. The new commit was not pushed to repo with a branch and so is lost. It can be recovered by looking into the reflog. Suggestion:

Only checkout tags in submodules. Let senior dev update and test changes in submodules. Also, submodules must be stand-alone projects that can be tested and developed separately.

Tips

Frequently Used Commands

git commit -a                     # Add all changes and commit in one pass
git commit --amend                # Amend tip current branch (message, add some files) - also for merge commits

Working the Git Way

Check project diff before commit -a:

git diff                          # First see what's in the working tree (or git status)
git commit -a                     # Commit all changes

Give git commit a directory argument instead of using -a:

git commit fs/                     # Commit all changes in directory fs

Clean up an ugly sequence of commits ([18]).

First make sure that the ugly sequence is on some temporary branch target (what we aim for), and that end result is good and clean.
Switch back to starting point, and do:

git diff -R target > diff             # diff to target

Edit diff file, to select only those changes we want to include in a first commit. Then do a git-apply diff

vi diff
git-apply diff                        # Must be in project root dir

Test, finalize the last changes before commits, and diff against target if necessary.

# test test test
git diff -R target > diff             # if necessary

Commit, and repeat from step 2.
When done, branch target can be removed

Use gitk to get a graphical visualisation of current commit, or some subsets. For instance

gitk                                 # View current commit and all ancestors
gitk master..                        # View changes to current branch (i.e. reachable from HEAD, excluding master)

Use git stash to save the current state of the working tree (see [19]).

git stash                            # Save current work in working tree
...                                  # (whatever, including git reset --hard...)
git stash apply                      # Bring back changes in working tree

Forgot to add some files in the previous commit? Mistyped the commit message? Use git commit --amend:

git commit                           # Oups! forgot one file
git add somefile                     # ... Add the missing file
git commit --amend                   # ... and replace the previous commit

Word-by-word diffs

Reference:

Blog [20] and [21]
Manpage gitattributes(5)
Manpage git-diff(1)

Add the following alias to your ~/.gitconfig file:

[alias]
    wdiff = diff --color-words
    wshow = show --color-words

Now you can have word-by-word diffs / shows:

   git wdiff fname
   git show

Improving tokenization in diffs

Diffs can even be further improved thanks to the use of the .gitattributes file in your repository. For instance:

*.tex   diff=tex
*.h     diff=cpp
*.c     diff=cpp
*.cpp   diff=cpp

One can also define filters that would clean files before checkin/checkout.

Show Git Branch in Prompt

Links:

The simplest is to add $(__git_ps1) to variable PS1 in file ~/.bashrc. Alternatively one can use \[\e[31m\]$(__git_ps1 " (%s)")\[\e[0m\] to prepad a space when branch name is displayed and use red color.

#PS1 in cygwin
PS1='\[\e]0;\w\a\]\n\[\e[32m\]\u@\h \[\e[33m\]\w\[\e[31m\]$(__git_ps1 " (%s)")\[\e[0m\]\n\$ '

Undoing / Reverting a commit

See [22] (stackoverflow.com)

Undoing last commit

The much simpler git commit --amend:

git add ...
git rm ...
git commit --amend          # Current index state will be used for new commit

The usual reset / add / commit sequence:

git commit ...
git reset --soft HEAD^      # Move back to previous commit, keep current changes
# edit edit edit...
git add ...                 # Add modified files
git commit -c ORIG_HEAD     # Commit, reedit old comment as base (-C to reuse w/o editing)

Reverting a past commit

Use git revert ...:

git revert -n HEAD             # Revert last commit, don't commit yet
git revert -n 1234567          # Revert a given commit, don't commit yet
git commit ...

Revert with Cherry Picking changes

The easiest is to reuse the patch method as explained by Linus above in Working the Git Way:

git show -b -R <commit> > diff   # Get a revert patch
vi diff                          # Edit the patch as necessary
git apply diff                   # Apply the patch

Housekeeping / compress repository

Repack repository:

git repack -adf
git repack -adF                  # Also repack already packed objects - rarely needed

# Complete sequence include garbage collect
rm -rf .git/refs/original/            # remove the old reference backup used for rebase
git reflog expire --expire=now --all  # tell git that all reflog objects expire now
git gc --prune=now                    # first prune everything
git gc --aggressive --prune=now       # ... then again and pack objects together

Squashing Merge Commits

Let's assume we have this situation, and we want to squash the 2 merge commits M1 and M2 (see [23]).

Initial situation		Final situation
X --- Y --------- M1 -------- M2 (my-feature) / / / / / / a --- b --- c --- d --- e --- f (stable)	→	X --- Y --------------------- M2' (my-feature) / / / / a --- b --- c --- d --- e --- f (stable)

Easy solution:

git checkout my-feature
git reset --soft Y
git rev-parse f > .git/MERGE_HEAD
git commit

Result is commit M2', identical to M2 above.

Finding files

Using git ls-files it is very easy and fast to locate a file in a git repository:

git ls-files                          # List all files in the repository
git ls-files *.c                      # List all files with .d suffix
git ls-files */gcc.mk                 # Find a specific file

Cloning to/from a Server using SSH

Reference: [24], [25], [26]

Standard peer-2-peer clone

The standard method is simply to use the clone command:

git clone user@server:repositories/myproject

This assumes one has SSH access to user@server.

Bare central repo - SSH access to client

If the clone is not meant to be used locally, but only as a central repository that other users will clone from, it is better to create a bare repository:

git clone --bare user@server:repositories/myproject myproject.git

By default, clone adds the source server as remote. This is usually not necessary:

cd myproject.git
git remote remove origin

Another method is to use git fetch in that case (which does not create a remote):

git init --bare myproject.git
cd myproject.git
git fetch user@server:repositories/myproject "*:*"          # Fetch from local repository

Bare central repo - restricted access from client

Usual, the central git server can't access to remote clients, and also, only offers restricted access from client (via SSH git-shell for instance). In that case, the method is to create an empty bare repository on the server (either via console or via some web interface):

ssh git@griffin
cd repositories
git init --bare myproject.git

And then, from the client, add the server as a remote and push all references:

git remote add git@griffin:repositories myproject.git
git push --all origin

Bare central repo - full access from client

In this method, we create a local bare copy, and then copy it to the remote server.

git clone --bare myproject myproject.git          # Create a bare clone of your repository, if not available yet
cd myproject.git
git remote remove origin
cd ..
scp -r myproject.git/ git@griffin:repositories/   # Copy the repository to server - requires full SSH access
rm -rf myproject.git                              # Delete local bare clone

Alternatively, one can transfer the repository using another user, but then access conditions and file ownership must be corrected.

See git-clone below for more details.

Cloning from a Server using SSH (limited access)

Reference: [27]

Say you have an SSH access to a server but git-core is not installed you you can't install it yourself (for instance on a shared hosting server). You can still use git but it requires some "hacking":

First copy the executables from package git-core, directory /usr/bin to some directory on the server where you have write access (say private/bin). Note that somes files are actually symlinks:

private/bin/

git*

git-receive-pack

git*

git-shell*

git-upload-archive

git*

git-upload-pack*

Clone from the server using -u command-line switch:

git clone -u </path/to/private/bin/git-upload-pack> user@server:private/git/myproject.git

Edit the local myproject/.git/config file to add the lines marked with a +:

[remote "origin"]
     fetch = +refs/heads/*:refs/remotes/origin/*
     url = user@server:private/git/myproject.git
+    uploadpack = /path/to/private/bin/git-upload-pack
+    receivepack = /path/to/private/bin/git-receive-pack

Using a remote with custom port

Just use the generic address ssh://username@hostname:port/...

git clone ssh://username@hostname:22222/home/git/repo/myrepo
git remote add origin ssh://username@hostname:22222/home/git/repo/myrepo

Cloning from a Server using `git:` Protocol over a Proxy

Reference: [28], [29]

The referenced links propose some script. Here another variant. Add this script to your path (say ~/bin/proxygit):

#!/bin/bash
# proxygit - git through http proxy
#
# Usage:  proxygit [options] COMMAND [ARGS]
#   Setup Git HTTP proxy variable GIT_PROXY_COMMAND and call git with the given parameters.
#   The proxy settings are read from env variable $http_proxy
#
# Note:
# - Requires package socat
# - $GIT_PROXY_COMMAND must not be defined

if [ -n "$GIT_PROXY_COMMAND" ]; then
	PROXY=$(echo $http_proxy|sed -r 's!(http://)?([^/:]*):([0-9]*)/?!\2!')
	PROXYPORT=$(echo $http_proxy|sed -r 's!(http://)?([^/:]*):([0-9]*)/?!\3!')
	exec /usr/bin/socat - "PROXY:$PROXY:$1:$2,proxyport=$PROXYPORT"
else
	export GIT_PROXY_COMMAND="$0"
	exec git "$@"
fi

Work with local and remote branches

Let's assume you have already a remote repository setup, like you would obtain if you clone a remote repository:

git clone user@server:repositories/project.git

By issuing this command, git will automatically:

Create a local repository, clone of the remote one
Call the remote repository origin
Create a local branch master, and
Configure it to track the branch master on origin. That remote branch is called locally origin/master.

The following commands can be used to create, track, delete local and remote branches.

# CREATE a local branch
git branch newbranch                          # Create a new local branch 'newbranch'
                                              # ... use "git checkout newbranch" to check it out

# CREATE & CHECKOUT a local branch
git checkout -b newbranch                     # Create a new local branch 'newbranch' and check it out

# PUBLISH a local branch (for TRACKing)
# git push [remotename] [localbranch]:[remotebranch]
git push origin serverfix                     # Push a local branch 'serverfix' to remote (create it if necessary)
git push -u origin serverfix                  # ... same but also set upstream reference for TRACKING
git push -u origin serverfix:serverfix        # ... same as above
git push -u origin serverfix:coolfix          # ... same but call the branch 'coolfix' on the remote

# TRACK a remote branch
git branch --track sf origin/serverfix        # Create a local branch 'sf' that tracks remote branch 'serverfix'
git branch --set-upstream sf origin/serverfix # ... same, but when local branch 'sf' already exists

# TRACK & CHECKOUT a remote branch
git checkout --track origin/serverfix         # Checkout a new local branch 'serverfix' to track remote branch 'serverfix'
                                              #   (remember that this branch is called locally 'origin/serverfix')
git checkout -b sf origin/serverfix           # ... same as above, but the local branch is named 'sf'

# FETCH / UPDATE from remote
git fetch
git fetch --prune                             # After fetching, remove any remote tracking branches that no longer exist on the remote

# FETCH from & MERGE with remote
git pull
git pull --prune                             # After fetching, remove any remote tracking branches that no longer exist on the remote

# DELETE a branch
git branch -d sf                              # Delete local branch 'sf'
git branch -d -r origin/serverfix             # Delete remote tracking branch 'serverfix'
git push origin :serverfix                    # Delete branch 'serverfix' on 'origin'
                                              # (basically this means push nothing to remote 'serverfix')

In summary:

Use git branch to create, update, delete branches on the local repository.
Use git checkout to checkout (possibly new) local branches.
Use git push to update the remote, possibly publishing or deleting branches.

Define a diff textconv filter

This applies a diff filter, but only for git diff and git log commands:

Edit file .gitattributes, add (! no quotes around the diff parameter)

*.adr       diff=bookmarks_adr

Edit file ~/.gitconfig, add

[diff "bookmarks_adr"]
    textconv = sed -r '/NAME=|URL=|SHORT NAME=/!d'

Duplicate a work tree at a given commit without duplicating the `.git` repo

This can be done with git archive, which is a bit like the svn export function of svn:

# create current subtree at version commit in ~/tmp/project/sub
# !!! DON'T FORGET *trailing slash* in --prefix
git archive --prefix project/sub/ commit . | tar -x -C ~/tmp   
# Same as above
git archive --prefix project/ commit sub | tar -x -C ~/tmp

Pushing to a non-bare repository

(see [30])

Let's consider we have

2 repositories,
one on a machine work and
another one on a machine home,
we cannot push to home because it is not visible

By default it is not possible to push to work because it is not a bare repository. However it is possible to push to another branch than the current one:

# On machine home, pushing to branch 'from-home' on work
git push work HEAD:from-home
# On machine work
git merge from-home

We can configure home to always push master to branch from-home on remote work:

git config remote.work.push +master:from-home

Fetch a single branch from a remote

Say you have a big remote repository, and you want to only fetch a single branch from it so that to save disk space. This is easily done as follows:

git remote add <remote> <url>
git fetch <remote> <remotebranch <localbranch>    # This will fetch branch <remotebranch> from <remote> and call it <localbranch> locally

Restore file timestamp

Creation / modification timestamps are not kept by Git, but one can set the file timestamp to e.g. commit timestamp using the following script (from [31]):

for FILE in $(git ls-files)
do
    TIME=$(git log --pretty=format:%cd -n 1 --date=iso $FILE)
    TIME=$(date -j -f '%Y-%m-%d %H:%M:%S %z' "$TIME" +%Y%m%d%H%M.%S)
    touch -m -t $TIME $FILE
done

Import commits from another repo easily

The easiest is to export the commits as patches with git format-patch, and then import then back.

cd repo1
git format-patch HEAD~3..
cd ../repo2
git am ../repo1/00*

Backup a GitHub repository

First create the repository [32]:

git clone --mirror git://github.com/user/project.git

Then to update:

git --git-dir project.git remote update

Change active branch in a bare repository

if you want the default branch to be something other than master, you need to do this [33]:

git symbolic-ref HEAD refs/heads/mybranch

Which will update the HEAD file in your repository so that it contains:

ref: refs/heads/mybranch

This is well documented in the git-symbolic-ref.

Create a self-runnable comment

Say you use the following script to remove all CRLF and trailing white spaces in your projects and commit the changes.

# Convert CRLF to LF (2x to get rid of CRCRLF)
ag -lt0 | xargs -0 dos2unix       # or 'ag --files-with-matches --all-text --print0 ...'
ag -lt0 | xargs -0 dos2unix
# Convert CR to LF
ag -lt0 | xargs -0 mac2unix
# Remove trailing blanks/tabs
ag -lt0 | xargs -0 sed -ri 's/[ \t]+$//'

We can store this script in the git comment, and later reuse that comment to run the script again. For this, format the commit text as follows:

Remove CRLF and trailing spaces (self-runnable comment)

 # Run this comment as follows (assuming 'origin/nows' points to this commit)
 #
 # git log -1 origin/nows; read -p "==== ^C to break, ENTER to execute ===="; source <(git log -1 --pretty='%b' origin/nows)

 # Convert CRLF to LF (2x to get rid of CRCRLF)
ag -lt0 | xargs -0 dos2unix       # or 'ag --files-with-matches --all-text --print0 ...'
ag -lt0 | xargs -0 dos2unix
 # Convert CR to LF
ag -lt0 | xargs -0 mac2unix
 # Remove trailing blanks/tabs
ag -lt0 | xargs -0 sed -ri 's/[ \t]+$//'

For easy retrieval, create a branch nows on the remote that points to that commit. Next time, to run the script again:

git log -1 origin/nows; read -p "==== ^C to break, ENTER to execute ===="; source <(git log -1 --pretty='%b' origin/nows)

Merge / cherry-pick with renames

If git cannot merge/cherry-pick because of renames, consider the following [34]:

# Too much changes
git merge -M25% _commit_                    # Lower rename threshold (default is 50%)
git cp -X-X rename-threshold=25% _commit_   # Idem with cherry-pick
# Too many files changed
git config merge.renameLimit 999999

use diffcore-pickaxe for detecting addition/deletion of specified string

Use diffcore-pickaxe:

git log -S<string>        # Detect filepairs with != number of occurences of <string>
git log -G<regex>         # Detect filepairs whose diffs contains an added/deleted line matching <regex>
git log --pickaxe-all ... # ... idem, but keep entire changeset

Improve diff output with `diff-highlight`

diff-highlight is contrib script shipped with git and that improves the regular diff output of git.

To install:

cp /usr/share/doc/git/contrib/diff-highlight/diff-highlight ~/bin
chmod a+x ~/bin/diff-highlight
git config --global pager.log 'diff-highlight | less'
git config --global pager.show 'diff-highlight | less'
git config --global pager.diff 'diff-highlight | less'

Alternatively, use a symlink to stay up-to-date:

ln -s /usr/share/doc/git/contrib/diff-highlight/diff-highlight ~/bin/
sudo chmod a+x ~/bin/diff-highlight

To get slightly better highlight colors:

git config --global color.diff-highlight.oldNormal "red bold"
git config --global color.diff-highlight.oldHighlight "red bold 52"
git config --global color.diff-highlight.newNormal "green bold"
git config --global color.diff-highlight.newHighlight "green bold 22"

For even better highlighting, consider using diff-so-fancy.

Rewrite Remote URL

git config --global url.ssh://git@github.com/.insteadOf https://github.com/

Reference (cache) repositories to speed up clones

git clone --mirror git://git.drupal.org/project/drupal.git ~/gitcaches/drupal.reference
git clone --reference ~/gitcaches/drupal.reference git://git.drupal.org/project/drupal.git

And the clone time is on the order of 2 seconds instead of several minutes. And yes, it picks up new changes that may have happened in the real remote repository. [35]

To go beyond this we can have a reference repository that has many projects referenced within it.

mkdir -p ~/gitcache.git                   # .git mandatory or recursive clone fails
cd ~/gitcaches.git
git init --bare
mkdir modules                             # --FIX-- recursive clone
for repo in drupal views cck examples panels  # whatever you want here
do
  git remote add $repo git://git.drupal.org/project/$repo.git
  ln -sf .. modules/$(basename $repo)     # --FIX-- recursive clone
done

git fetch --all --tags                    # Can be run at regular interval (optional)
                                          # --tags necessary to get *all* commits

Now project can be cloned rapidly like

git clone --reference ~/gitcache.git git://git.drupal.org/project/drupal.git

If the project contains submodule:

git clone --reference ~/gitcache.git git://git.drupal.org/project/parent.git
cd parent
git submodule init
git submodule update --reference ~/gitcache.git

Detaching from alternates

When planning to move or delete the reference repository, first detach from it:

git repack -a
git submodule foreach git repack -a
find $(find . -name .git) -name alternates # Make sure this finds only the right files
find $(find . -name .git) -name alternates | xargs rm  # Remove the alternates files

Cloning projects with submodules using clone --recursive

Note that to use recursive clone (git clone ... --recursive):

Cache repo must end with suffix .git (ie. gitcache.git, not gitcache)
Symlinks modules/... must exist.
Submodules name must match the name of the repo.

Otherwise the workaround with symlinks will fail.

To clone recursively:

git clone --reference ~/gitcache.git git://git.drupal.org/project/parent.git --recursive

Refilling incomplete projects

This is necessary if the cache is moved or deleted [36]. Incomplete projects can be refilled with

git repack -a
git submodule foreach git repack -a
find $(find . -name .git) -name alternates # Make sure this finds only the right files
find $(find . -name .git) -name alternates | xargs rm  # Remove the alternates files

Reduce timeout when host unavailable (SSH)

Use environment variable GIT_SSH_COMMAND (see man git):

GIT_SSH_COMMAND='ssh -o ConnectTimeout=3' git fetch ...

Fetch all git repositories (cron job)

Example of cron job entry (inspired from SE):

42 * * * *      savelog -n ~/log/git-fetch.log; \
parallel -i chronic sh -c "date; echo {}; cd {}; GIT_SSH_COMMAND='ssh -o ConnectTimeout=3' git fetch --all" -- \
$(locate .git/config | sed '/\/.vim\/plugged\//d; /\/.cargo\//d; s_/.git/config__') >> ~/log/git-fetch.log 2>&1

Limitations:

Error when connecting to unavailable hosts
Fail if space in paths. Could it be fixed by sed -r 's/ /\\ /g'?
Does not skip samba shares (samba used to leave awful zombie processes if interrupted).

Mirroring a remote repository

To create a local mirror:

git clone --mirror <firsturl>

# To update local mirror
# [https://stackoverflow.com/questions/6150188/how-to-update-a-git-clone-mirror]
cd <project>
git remote update
# git remote update --prune

To create a distant mirror:

git clone --mirror <firsturl>
cd <project>
git remote rm origin
git remote add origin <secondurl>
git push --mirror origin

# To update distant mirror
cd <project>
git push --mirror origin

Review and merge GitHub pull request

Step 1: From your project repository, check out a new branch and test the changes.

git checkout -b maeveynot-window-is-long master
git pull https://github.com/maeveynot/xseticon.git window-is-long

Step 2: Merge the changes and update on GitHub.

git checkout master
git merge --no-ff maeveynot-window-is-long
git push origin master

List large files in git history (git-ls-large)

Inspired from the powerful script from SO:

#! /bin/bash
#
# git-ls-large

# Gets the command name without path
cmd(){ echo `basename $0`; }

# Error message
error(){
    echo "`cmd`: invalid option -- '$1'";
    echo "Try '`cmd` -h' for more information.";
    exit 1;
}

# Help command output
usage(){
    column -t -s ";" << __USAGE__
`cmd` [OPTION...]
    -h, --help; Print this help
    -M, --mega; Ignore files less than 1MB
    -r, --raw; Don't sort nor format output
    -x, --exclude COMMIT; Exclude files present in COMMIT
__USAGE__
    exit $1
}

# Parse options
OPTS="$(getopt -o hMrx: -l help,mega,raw,exclude: --name "$(basename "$0")" -- "$@")"
[ $? -eq 0 ] || usage 1

eval set -- "$OPTS"
unset OPTS

EXCLUDE=
MEGA=false
RAW=false

while true
do
    case $1 in
        -h | --help )    usage 0 ;;
        -M | --mega )    MEGA=true; shift ;;
        -r | --raw )     RAW=true; shift  ;;
        -x | --exclude ) EXCLUDE=$2; shift; shift ;;
        -- )             shift; break ;;
        * )              error $1 ;;
    esac
done

mega(){
    if $MEGA; then
        awk '$2 >= 2^20'
    else
        cat
    fi
}

exclude(){
    if [ -n "$EXCLUDE" ]; then
        grep -vF --file=<(git ls-tree -r $EXCLUDE | awk '{print $3}')
    else
        cat
    fi
}

format(){
    if ! $RAW; then
        cut -c 1-12,41- \
        | $(command -v gnumfmt || echo numfmt) --field=2 --to=iec-i --suffix=B --padding=7 --round=nearest
    else
        cat
    fi
}

git rev-list --objects --all \
| git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' \
| sed -n 's/^blob //p' \
| exclude \
| mega \
| sort --numeric-sort --key=2 \
| format

Alternative script: see Magnetikonline (GitHub).

List parent or children branches of a given commit

Parent branches

The git branch command now provides two subcommands to list "parent" branches:

git branch --merged master         # List branches whose tip is reachable from master
git branch -a --merged master      # ... same, incl. remote branches
git branch --no-merged master      # List branches whose tip is NOT reachable from master

The same subcommands are available for git tag:

git tag --merged master         # List tags that are reachable from master
git tag --no-merged master      # List tags that are NOT reachable from master

Child branches

Create the following script git-list-children, and save it somewhere in the user path:

#! /usr/bin/perl
#
# From http://stackoverflow.com/questions/2208948/display-all-first-level-descendant-branches-using-git
# Extended by Michael Peeters
# 
# Usage: git list-children [--first] [<branch>]

use warnings;
use strict;

my $firstchildonly=0;
my $HEAD="HEAD";

sub refs {
  open my $fh, "-|", "git", "for-each-ref",
                            "--format=%(objectname)\t%(refname:short)"
    or die "$0: failed to run git for-each-ref";

  my %ref2sha;
  while (<$fh>) {
    chomp;
    my($sha,$ref) = split /\t/;
    $ref2sha{$ref} = $sha;
  }

  \%ref2sha;
}

sub is_child {
  my($ref) = @_;

  # git rev-list ^dev master
  my $refs = `git rev-list ^$ref $HEAD -- 2>&1`;
  die "$0: git rev-list-failed.\n$refs" if $?;

  $refs !~ /\S/;
}

# Parse command options (--firstchild)
while ( $#ARGV>=0 ) {
    if( $ARGV[0] =~ m/^\-/ ) {
        $firstchildonly=1 if $ARGV[0] eq "--first";
    }
    else {
        $HEAD=$ARGV[0];
    }
    shift @ARGV;
}

chomp(my $head = `git rev-parse $HEAD 2>&1`);
die "$0: git rev-parse failed.\n$head" if $?;

my $ref2sha = refs;
my %headsha = reverse %$ref2sha;

REF:
foreach my $ref (keys %$ref2sha) {
  my $refsha = $ref2sha->{$ref};

  next if $refsha eq $head || !is_child $ref;

  if ($firstchildonly) {
      my @log = `git log --pretty=format:%H ..$ref 2>&1`;
      die "$0: git log failed.\n@log" if $?;
      for (@log) {
        chomp;
        next if $_ eq $refsha;
        next REF if exists $headsha{$_};
      }
  }

  print $ref, "\n";
}

Fast-forward a branch without checkout

From SO answer:

git fetch origin master:master            # Will fail if local master can't be ff to origin/master
                                          # Note: this is remotebranch:localbranch
git fetch . foo:master                    # FF local master to local foo
                                          # Note: only localbranch is updated

Note the handy notation . to denote the local repository as a remote repository for fetch.

This could be made into an alias:

 [alias]
     sync = !sh -c 'git checkout --quiet HEAD; git fetch upstream master:master; git checkout --quiet -'

This alias will sync the master branch with the corresponding upstream branch.

View commits that belong to a given branch only

Inspired from an answer on Stack Overflow:

# This excludes tags
git log heads/mybranch --not --exclude=mybranch --branches --remotes

# ... a longer variant using git rev-list (using useful option --no-walk)
git log --branches --remotes --not $(git rev-list --no-walk --exclude=mybranch --branches --remotes)

# Including tags
git log --all --not --exclude=refs/heads/mybranch --exclude=HEAD --all

This will basically show the commits that would be lost (after garbage collection) if the given branch is deleted. In the first example, heads/ is needed in the case there is a tag that unfortunately has the same name as given mybranch.

This can be made into an alias:

# For Git 1.22+
git config --global alias.only '!b=${1:-$(git branch --show-current)}; git log --oneline --graph "heads/$b" --not --exclude="$b" --branches --remotes #'
# For older Git:
git config --global alias.only '!b=${1:-$(git symbolic-ref -q --short HEAD)}; b=${b##heads/}; git log --oneline --graph "heads/$b" --not --exclude="$b" --branches --remotes #'

Or the variant that includes all the tags except HEAD (adapt as above for older Git):

git config --global alias.only '!b=${1:-$(git branch --show-current)};  git log --oneline --graph --all --not --exclude="refs/heads/$b" --exclude=HEAD --all #'

Or the variant that includes all the tags including HEAD (and removing current branch as default since it won't output anything)

git config --global alias.only '!git log --oneline --graph --all --not --exclude=\"refs/heads/$1\" --all #'

To use remote branch, the alias must be adapted a bit:

git config --global alias.only '!git log --oneline --graph "$1" --not --exclude="$1" --remotes --branches #'

Use alias with parameters

From Stack Overflow, use ! at the beginning of an alias to tell git the alias is a shell script:

[alias]
    files = "!git diff --name-status \"$1^\" \"$1\" #"

Note the final # to comment out all the parameters that git adds (this can be checked with GIT_TRACE=2 git files a b c d). Note also the importance of quoting in general when dealing with filenames.

Some may find wrapping in a function more elegrant

[alias]
    files = "!f() { git diff --name-status \"$1^\" \"$1\"; }; f"

Bash Script-Fu

Some tips on how to use Git in scripts.

Plumbing, porcelain and `--porcelain`

Best answer is from VonC on SO:

Use plumbing commands as building blocks for scipts.
Porcelain commands are more user-friendly commands that are not meant to be used within scripts.
Some of these commands have however a --porcelain flag that produce a stable and simple output easily parseable by scripts.

Get commit of current or any branch

git rev-parse HEAD
git rev-parse branchname

Get current branch name

There are various options (see [37]).

# If not on a branch
git rev-parse --symbolic-full-name --abbrev-ref HEAD    # HEAD
git symbolic-ref -q --short HEAD                        # nothing, fails
git symbolic-ref -q --short HEAD || echo HEAD           # HEAD
git branch --show-current                               # nothing, exit code 0

# If tag "master" exists
git rev-parse --symbolic-full-name --abbrev-ref HEAD    # heads/master
git symbolic-ref -q --short HEAD                        # heads/master
git branch --show-current                               # master

# If no tag "master"
git rev-parse --symbolic-full-name --abbrev-ref HEAD    # master
git symbolic-ref -q --short HEAD                        # master
git branch --show-current                               # master

git branch --show-current requires Git 2.22 (2019 Q2).

Get `.git` directory location

We can use function __gitdir, but it requires to be exported first. Add to ~/.bashrc or ~/.profile:

export -f __gitdir

Alternatively one can also use [38]:

GITDIR=$(git rev-parse --git-dir)

Get root directory location

GITROOT=$(git rev-parse --show-toplevel)

List all references

#List all references that starts with "refs/remotes/tags"
git-for-each-ref refs/remotes/tags

#Loop on these references
git-for-each-ref refs/remotes/tags | cut -d / -f 4- |
while read REF
do
  echo $REF
done

Detect whether current tree is dirty

From SO) and SO:

Using plumbing:

# Detect changes
git diff-index --quiet --cached HEAD || echo staging changes
git diff-index --quiet HEAD          || echo Tree has no difference with HEAD
# Note: 2nd reports no difference if there are difference in tree than cancels changes in index

# Detect untracked + ignored
git ls-files --other --error-unmatch . >/dev/null 2>&1; ec=$?
# ec = 0       untracked
# ec = 1       no untracked
# ec = other   an error occured

# Detect untracked only (exclude ignored files)
git ls-files --other --exclude-standard --error-unmatch . >/dev/null 2>&1; ec=$?

using --porcelain:

test -n "$(git status --porcelain)"   # Test if there are changes (but ignore ignored files)

How to get the latest tag name in current branch in Git?

From SO:

git describe --abbrev=0 --tags
if TAG=$(git describe --exact-match --abbrev=0 --tags 2>/dev/null); then
  echo This commit has tag '$TAG'
else
  echo This commit has no tag
fi

Tools

Git-Annex

git-annex allows managing files with git, without checking the file contents into git. While that may seem paradoxical, it is useful when dealing with files larger than git can currently easily handle, whether due to limitations in memory, time, or disk space.

Gitolite

A very simple but powerful Git server that enables access control via ssh.

gitolite manual

Install

We follow the very clear guide on Vultr.com:

Install dependencies and create a special user git:

apt-get update && apt-get upgrade && apt-get install git perl
useradd -m git
passwd git

Make sure we have our key on the server. If not, from the client:

scp ourkey.pub git@ourserver.org:ourname.pub

Install Gitolite on the server, directly from GitHub to get latest version. Before that, make sure that ~/.ssh/authorized_keys is empty.

su git
cd
rm .ssh/authorized_keys
git clone git://github.com/sitaramc/gitolite
mkdir -p $HOME/bin
gitolite/install -to $HOME/bin

Make ourself an administrative user:

bin/gitolite setup -pk ourname.pub

Make sure that user git is allowed by ssh daemon:

grep AllowUsers /etc/ssh/sshd_config
# AllowUsers root immie sshproxy git

Installation is complete. We can now administer everything from the client. For this, we clone the repository gitolite-admin:

Clone gitolite-admin on the client:

git clone git@ourserver.org:gitolite-admin

Add user by adding user key file in directory ./keydir.

cd gitolite-admin
cp whatever/user1.pub keydir/user1.pub

Add new repositories and configure access conditions by editing the file ./conf/gitolite.conf

vi conf/gitolite.conf

Commit the changes and push the new settings to the server

git add -A
git commit -m "Adding user1 and new repos"
git push

Add a new repository

Add the new repo in ./conf/gitolite.conf, with at least one user having some access, and push the changes to the server.
Gitolite will then create an empty repo we can clone on the client.

Remove a repository

Remove the repo from ./conf/gitolite.conf, and push to the server.
Log into the server, and delete the repository

ssh ourserver.org
su git
rm -rf ~/repositories/my-old-repo.git

Rename / move a repository

First rename / move the repository on the server

ssh ourserver.org
su git
cd ~/repositories
mv my-old-name.git my-new-name.git

Edit ./conf/gitolite.conf, and push to the server.

Add existing repo to Gitolite

Before that, make sure there is no hook in the depo (see gitolite manual).
Move the repo to ~/repositories directory:

scp -r my_existing_repo.git git@ourserver.org:repositories/

Make sure that

Repo are bare and have .git extension.
The git user must have ownership and write access.
There are no symbolic links leftovers.

On the server,

su git
cd
gitolite setup                 # No write access control if we forget this!

On the client, add the repo to ./conf/gitolite.conf. For wildcard repo, check the manual.

vi conf/gitolite.conf
git commit -am "Adding existing repo"
git push

Troubleshoot

Out of memory - fetch

I get the following message under cygwin (WinXP SP3, cygwin 1.7.9-1, git 1.7.5.1-1, perl 5.10.1-5), while doing a git svn fetch:

git svn fetch
# . . .
# . . .
# Out of memory during "large" request for 268439552 bytes, total sbrk() is 311015424 bytes at /usr/lib/perl5/vendor_perl/5.10/Git.pm line 908, <GEN1> line 615.

Solution is to increase Cygwin's maximum memory. Setting /HKLM/Software/Cygwin/heap_chunk_in_mb to 1024 (1GB) fixed the problem. Indeed Cygwin can only allocate 384MB for a process, and process ended up using more than 480MB... (see also [39])

Out of memory - gc / repack

$ git gc --prune --aggressive
Counting objects: 58668, done.
Delta compression using up to 2 threads.
fatal: Out of memory, malloc failed (tried to allocate 65795137 bytes)
error: failed to run repack

Warning! — Seems that git gc --aggressive has it's own settings for repack window. See git-gc.

Settings / configuration we can play on to help in solving that problem:

pack.window and pack.depth, which control how objects are stored using delta compression. window basically defines how far repack will look for making deltas, and depth controls the depth when nesting deltas.
pack.windowmemory will automatically tune down the values of pack.window and pack.depth when the memory limit is reached. This helps for repositories with both big and small files, allowing to still benefit from a large window with small objects while controling memory usage for big objects.
core.bigFileThreshold set the maximum size of objects manipulated for repack. Any object bigger than this threshold will not be repacked and simply stored deflated.
pack.threads set the number of threads used by repack (it seems that memory limit is set per thread)
Increase the amount of virtual & physical memory
Use a 64-bit OS

Settings examples:

My settings (got an out of memory - suboptimal pack warning during git gc --prune --aggressive though):

pack.windowmemory = 700m                # Default 0 (no limit)
pack.packsizelimit = 1g                 # Default unlimited
# No pack.threads settings
core.bigFileThreshold = 256m            # Default 512m

Tim's Settings:

pack.windowmemory = 256m
pack.packsizelimit = 1g
pack.threads = 1
core.bigFileThreshold = 256m

Settings on QNAP (ARM-based NAS):

sudo git config --system pack.windowmemory 128m
sudo git config --system pack.packsizelimit 1g
sudo git config --system pack.threads 1
sudo git config --system core.bigFileThreshold 256m

Some references:

git-repack manual page
git-config manual page
reduce threads [40]
reduce repack window [41])

Broken Pipe error code 13 on `git svn fetch`

This error is most likely due to network disconnection. Solution so far is to fetch repeatedly in a for loop:

for i in 1..50 ; do git svn fetch; done

Corporate SSL proxy

Cloning a repository over a HTTPS through an corporate proxy that intercepts SSL request will fail with the error:

git clone https://github.com/kien/ctrlp.vim.git ctrlp.vim
# Cloning into 'ctrlp.vim'...
# fatal: unable to access 'https://github.com/kien/ctrlp.vim.git/': SSL certificate
# problem: unable to get local issuer certificate

The solution:

Disable SSL certificate verification [42]

git config --global http.sslVerify false

Add corporate certificate [43].

server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none

hostname=XXX
port=443
trust_cert_file_location=`curl-config --ca`
sudo bash -c "echo -n | openssl s_client -showcerts -connect $hostname:$port 2>/dev/null  | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' >> $trust_cert_file_location"

See details on SO.

MinGW - Error: ssh_askpass: exec(/usr/lib/ssh/ssh-askpass): No such file or directory

This may occur when using git from another MinGW environment than the one provided with Git Windows. The fix is (i) to make sure your ssh agent works correctly, and (ii) to add the following line to ~/.bashrc:

export GIT_SSH=/usr/bin/ssh

To test whether your ssh-agent works correctly, just try to connect to the remote:

ssh gitolite@mygitserver.org

The connection should be made without prompting for any password.

Troubleshoot SSH

GIT_SSH_COMMAND="ssh -vvv" git clone ssh://git@<bitbucket URL>:<bitbucket port>/<project key>/<repository slug>.git

Troubleshoot bash error with __git_ps1

In some version of Bash, using __git_ps1 may gives strange error:

PS1='$(__git_ps1)\n'

# bash: command substitution: line 1: syntax error near unexpected token `)'
# bash: command substitution: line 1: `__git_ps1)'

The fix is to replace \n with \012 [44]:

PS1='$(__git_ps1)\012'

@@ Line 2,294: / Line 2,294: @@
 <source lang=bash>
 # Detect changes
-git diff-index --quiet --cached HEAD && echo staging changes
+git diff-index --quiet --cached HEAD || echo staging changes
-git diff-index --quiet               && echo tree has changes
+git diff-index --quiet HEAD          || echo Tree has no difference with HEAD
+# Note: 2nd reports no difference if there are difference in tree than cancels changes in index
-git diff-index --quiet HEAD          && echo Tree has no difference with HEAD
-# Note: 3rd reports no difference if there are difference in tree than cancels changes in index
 # Detect untracked + ignored

Git: Difference between revisions

Revision as of 01:32, 4 April 2021

References

Git SVN

Introduction

Distributed

Reliability

Performance

Terminology and Concepts

Install

From packages

From sources

In MSYS2

Configuration

General

gitk and tcl/tk

Hook - prevent non-fast-forwarded update

Install / Configure git-daemon

CR/LF conversion

Install Git Server with SSH / git-shell

GitHub repositories

Commands

Frequently used & common command options

git-ls-tree

git-verifypack

Submodules

Tips

Frequently Used Commands

Working the Git Way

Word-by-word diffs

Show Git Branch in Prompt

Undoing / Reverting a commit

Housekeeping / compress repository

Squashing Merge Commits

Finding files

Cloning to/from a Server using SSH

Cloning from a Server using SSH (limited access)

Using a remote with custom port

Cloning from a Server using git: Protocol over a Proxy

Work with local and remote branches

Define a diff textconv filter

Duplicate a work tree at a given commit without duplicating the .git repo

Pushing to a non-bare repository

Fetch a single branch from a remote

Restore file timestamp

Import commits from another repo easily

Backup a GitHub repository

Change active branch in a bare repository

Create a self-runnable comment

Merge / cherry-pick with renames

use diffcore-pickaxe for detecting addition/deletion of specified string

Improve diff output with diff-highlight

Rewrite Remote URL

Reference (cache) repositories to speed up clones

Reduce timeout when host unavailable (SSH)

Fetch all git repositories (cron job)

Mirroring a remote repository

Review and merge GitHub pull request

List large files in git history (git-ls-large)

List parent or children branches of a given commit

Fast-forward a branch without checkout

View commits that belong to a given branch only

Use alias with parameters

Bash Script-Fu

Plumbing, porcelain and --porcelain

Get commit of current or any branch

Get current branch name

Get .git directory location

Get root directory location

List all references

Detect whether current tree is dirty

How to get the latest tag name in current branch in Git?

Tools

Gitolite

Troubleshoot

Out of memory - fetch

Out of memory - gc / repack

Broken Pipe error code 13 on git svn fetch

Corporate SSL proxy

server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none

Cloning from a Server using `git:` Protocol over a Proxy

Duplicate a work tree at a given commit without duplicating the `.git` repo

Improve diff output with `diff-highlight`

Plumbing, porcelain and `--porcelain`

Get `.git` directory location

Broken Pipe error code 13 on `git svn fetch`