Notes on Using GIT

Magit comes closest to being ideal for using Git in Emacs.

Magit works extremely well for creating proper per-logical-change change sets from a large collection of wide-ranging changes in the local working directory.

Diffs can be refined into multiple hunks by changing the number of lines of context, and diff hunks can be individually included (staged) in a commit.

You can easily¹ test a partial commit in isolation from other unrelated changes too, using the stash feature.

1. First stage changes (files, or hunks of diffs to files) for the commit (press s on a file or hunk). Then stash everything (press z) but with the --keep-index option (press x instead of z again). It may be a good idea to stash un-tracked files too, typing -u (or -a to also stash ignored files) before typing x.
2. Test the commit in isolation, commit it when it is ready (just don't make any unrelated changes when fixing anything for the commit). Remember to stage these new changes. To commit the staged changes press c c, edit the commit message, press C-c C-c. Finally pop the stash (press A with the cursor on its line).

Note: even though the changes in the index, i.e. in the staging area, and eventually in the commit, are included in the stash, they will not be applied when the stash is popped again after the commit is completed.

git cvsimport -v [-i|-C dir_to_create] [-k] [-u] -d $CVSROOT cvs_module_to_checkout

• -i will import only, like git clone --bare (but I think it assumes an existing ./.git directory), or with -C will create a working directory with a checked out copy.
• -k will kill keywords to prevent confusion, as with CVS's -kk
• -u will translate underscores in tags back into the more desirable .

You may need to set CVSREADONLYFS=1 in the environment for local access to a read-only repository.

For local repository access the CVS modules file is ignored. You must use the full sub-directory name for the module within the CVS repository.

Create a clone of the Git repository that was imported from CVS so that you can work on it:

cd /work/$USER
git clone ../cvs_module dir_to_create

check out the CVS project into a desired working directory

cvs -d $CVSROOT co -d ~/work/project project

• add .git and .gitignore to ~/.cvsignore
• add CVS to .gitignore
  • XXX or maybe not – some people keep these in Git so that they can then clone the Git repo and still use CVS in that new clone, which makes a lot of sense sometimes, e.g. for making patches or seeing old CVS history, etc., etc.

You'll want the Emacs VC tools to operate on the local Git repository, not on the CVS repository:

;; Move git backend in front of cvs backend
;; (Emacs manual, Chapter 32, esp. 32.1.8.2 Local Version Control)
(setq vc-handled-backends
   (append
    (list 'Git)
    (delq 'Git vc-handled-backends)))

(need some way to make this work per working directory, not globally)

git checkout -t master -b local-hacks

It is possible to create any number of local branches in git, try things out etc.

git commit -am "A new local hack!"

When someone using CVS updates something in the CVS repository you can simply switch your local repository back to the 'master' branch (git stash; git checkout -b master), then run cvs update, and finally commit the update to the git master branch (git commit -a), then switch back to what you were doing (git checkout -b local-hacks; git stash pop). When ready you can also merge updates from the master branch into your local branch (git merge master or git rebase master as appropriate);

First off merge local work to master:

git checkout master
git merge local-hacks

Then update from CVS to merge central work with local work in CVS:

cvs update -d -p
# merge any conflicts!
# (don't need to commit if there are no changes from CVS)
git commit -am "Sync with CVS"

Verify and review what was done locally

git log cvssync..
cvs diff
# ... all is OK?  Push to CVS:
cvs commit -m 'Push local-hacks blabbety blah'
# iff there are Id tags, they will have changed
git commit -am "Sync CVS Id tags"
git tag -f cvssync

First we create a local branch called local which "tracks" a remote branch (in this case the one called master in the repository from which we cloned our local copy), and we set the default action on our new local branch to rebase by default instead of merge. I.e. when we do a git pull in the future our local branch will be rebased onto the head of the origin/master branch (instead of the origin/master branch being merged to the local branch).

git checkout -t -b local origin/master
git config branch.local.rebase true

master is the default name of the trunk of the project revision tree. Any branch in the remote project can be checked out to be worked upon, for example a release branch. If you work on a non-trunk branch you might want to choose a local branch naming convention that reflects which origin branch it is rooted on (i.e. tracking), e.g., by pre-pending local- to the original branch name, like this:

git checkout -t -b local-v1.0 origin/v1.0
git config branch.local-v1.0.rebase true        # "-v1.0"???

Not all projects use release branches though, some just use tags, in which case if you set the starting point of your local branch to a release tag then the that will not ever be a moving target (assuming the upstream tag never moves) and so the use of "rebase" operations to track the remote release will not be necessary.

You would normally work on your local branch; there's no need to switch back to the master branch.

git add Makefile
git commit -m 'My local Makefile configurations'

You use git add filename because you are adding new content to be tracked. This is also called "staging". The current content of the modified file(s) is added to the "INDEX".

You can delay git commit until you've added (staged) multiple related content changes. You do not specify filenames to the commit command.

git commit only commits added (staged) changes by default, not all modifications to the working tree. However git commit -a will commit all modifications in the current tree (by implicitly doing the "add" operation first, literally as if git add -u were run).

With your local branch configured to use "rebase" (instead of the default "merge" action) now each time you pull new commits from the distribution repository your local commits will be rewound and stashed temporarily, the local branch point will be moved to the new head of the origin/master branch, and the stashed changes from your local branch will be replayed on top of the new branch point.

git fetch           # \
                    # -> fetch + {merge,rebase} == "git pull"
git rebase          # /

Note that if you do the git fetch first then you can look at the changes which will be pulled:

git diff master origin/master
# or just (assuming you are on the master branch already)
git diff origin/master
# or alternately:
git diff ...origin

You may need to move some/all of your local changes to another branch in the remote project, either to upgrade/downgrade your local changes to another release branch, or perhaps to move your changes to the trunk so that they can be submitted upstream.

Basically this is done by copying the changes from one branch (your local branch, for example) to another, while of course leaving them intact and unchanged on the original branch.

The .git/config file should look a bit like this:

[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[branch]
	autoSetupRebase = always
[pull]
	rebase = true
[remote]
	pushDefault = robohack
[magit]
	gh-pulls-repo = wanderlust/apel
[remote "origin"]
	url = https://github.com/wanderlust/apel
	fetch = +refs/heads/*:refs/remotes/origin/*
	# also fetch any pull requests (the github way)
	fetch = +refs/pull/*/head:refs/pull/origin/*
	# also fetch any pull requests (the magit/forge way)
	fetch = +refs/pull/*/head:refs/pullreqs/*
[branch "apel-wl"]
	remote = origin
	merge = refs/heads/apel-wl
        mergeOptions = --ff-only
	rebase = false
[remote "robohack"]
	url = git@github.com:robohack/apel.git
	fetch = +refs/heads/*:refs/remotes/robohack/*
	# also fetch any pull requests (both ways)
	fetch = +refs/pull/*/head:refs/pull/robohack/*
	fetch = +refs/pull/*/head:refs/pullreqs/*
[branch "local"]
        description = "Local integration branch."
	remote = robohack
	merge = refs/heads/local
	rebase = true

GitHub's own "hub" tool, written in Go:

git clone https://github.com/github/hub f-hub
cd f-hub
hub fork

This tool uses your github user name as the useful remote name, while leaving the original project with the remote name "origin". Perhaps this makes more sense than using the confusing "upstream".

see:

https://help.github.com/articles/syncing-a-fork/

[ … ToDo: write this up properly in detail … ]

magit-status
create a stash (z z) [or otherwise kill unstaged changes -- e.g. reset hard (X h)]
Fetch all (f a) [git fetch --all] or better yet [git up], or even [git clone --mirror]
checkout (the local remote-tracking) master (b b) [git checkout master]
merge from upstream (F u) [git merge -ff-only upstream/master]

[ … there's also a theory that you might want a merge commit, so "-no-ff" … ]

update master on robohack, i.e. push to github fork (P p) [git push robohack master]
checkout your local branch (b b local) [git checkout local]
merge from master (m m) [git merge master]

[ … or should that normally always be (r e) [git rebase master local] ??? … ]

[[ fix, commit, etc. -- make sure local changes are committed! ]]
update 'local' on robohack, i.e. push to github fork (P p) [git push robohak local]

N.B.: Note that some(most?) upstream projects would rather the local changes be rebased onto the tip of their current development.

N.B.: be very careful not to "pull upstream" in the local branch! (only merge) [ … or "git rebase master local" ??? … ]

N.B.: leave the "Unmerged into origin/master" changes alone as they will have to go through upstream/master via a pull request to ever appear in origin/master

THIS IS PROBABLY NOT A GREAT IDEA!!! You cannot easily maintain local changes on an evolving branch, even if you rebase your changes to the HEAD after every pull from the upstream branch. Your local changes will forever appear to need pushing to the upstream even though you will not be permitted to do that.

Some (many?) GitHub public repositories have their own web "home" page at https://author.github.io/project.

However the default "fork" action on GitHub does not copy the "gh-pages" branch, and thus the fork does not automatically get its own web page based on the origin.

To copy this branch you can (after stashing any uncommitted changes):

git checkout gh-pages upstream/gh-pages
git push gh-pages origin

Then go to your forked repository on github and configure it under the "Settings" tab at the "GitHub Pages" section by selectin the source as the "gh-pages branch".

You will then probably have to make a change to one of the files on that branch and push it in order to get GitHub to actually create and publish the web page(s).

mkdir fooproj
cd fooproj
git init
echo 'ref: refs/heads/local' > .git/HEAD
pax -rz -s '/^fooproj-1.0//' -f ~/Downloads/fooproj-1.0.tgz
git add .
git commit --author='VenDor' -m 'Import of fooproj version 1.0'
git checkout -b vendor
git tag fooproj-1.0
git checkout local

Ideally, each self-contained feature is developed on its own appropriately named branch. When it is reasonably stable, it can be merged into one of the long-lived branches.

Creating a "topic" or "feature" branch, then merging that branch to each of the other branches which require the same changes (e.g., to each supported release) is the ideal way to manage more complex local changes, especially ones which are intended to be pushed upstream for eventual inclusion in the origin/master branch by upstream project managers.

Some folks suggest always using the "Branch Early and Often" methodology with Git, i.e. avoid doing any development ever on the master (trunk) branch – just do merges there. A branch in Git is in some ways the moral equivalent of a patch. One can periodically create a testing branch onto which all development ("topic" or "feature") branches are merged to see if everything works together; deleting the testing branch when done with each iteration of QA testing; and then merging all of the tested and verified "topic" or "feature" development branches to the master branch.

"Feature" branches should be forked from the oldest locally supported release branch (not the local branch forked from the original release branch, but rather just the original release branch itself), since the feature/fix they represent should not initially depend upon newer changes in newer releases (or in the trunk) as otherwise supporting the "topic" branch on the older releases may become too difficult.

Naming these branches with a feature/ prefix can help distinguish them from other long-lived branch names.

For features intended only for new, as-yet un-branched, releases it can be common that these branches will be rebased onto the HEAD of the current development or master branch frequently. In order to distinguish these branches from long-lived branches they can be marked as being "works-in-progress", e.g., wip/shiny-feature. After a rebase such branches will of course have to be pushed with git push --force-with-lease wip/shiny-feature. A wip/ prefix on your branch name will indicate that it is likely not a good idea to branch off that branch, because its history might change completely at any time.

A "topic" branch can be created on whichever branch is most easily worked upon in the local context. Great care must be taken though to make sure that the changes created to implement the new fix or feature do not depend on any other local changes and only depend on content from the remote repository – i.e. new features or fixes to be promoted upstream must be designed to work without any other local changes.

git checkout local-mynewfeature-v1.0 v1.0
# edit-compile-test-commit-loop

Now merge the new feature branch onto the local (previously created, see Create a local branch for your changes from the origin/master branch above) release branch:

git checkout local-v1.0
git merge local-mynewfeature-v1.0 --no-ff

Also merge it onto other locally-supported branches:

git checkout local-v2.0
git merge local-mynewfeature-v1.0 --no-ff

Finally merge it onto the local branch (also previously created) which tracking the remote trunk (master) so that it can be tested as a change set to be sent upstream:

git checkout local
git merge local-mynewfeature-v1.0 --no-ff
git format-patch -n HEAD~

The easiest way to see all the changes on a given branch is to switch to that branch and then run git diff with the name of the ref that specifies the base point of the branch. For example if a branch local-v2.0 was created from a release tag/branch v2.0 then you simple switch to the branch and run the diff:

git checkout local-v2.0
git diff v2.0

If you don't want to change the state of the working directory you can also do the diff by specifying the entire range of the branch:

git diff v2.0..local-v2.0

The easy way to move all the changes from one branch to another is of course to use git merge to merge them into a commit on the new branch. Here's an example of moving all the changes from the local branch to a new local-v2.0 branch created from the reference v2.0 release branch/tag:

git checkout -b local-v2.0 v2.0
git merge local --no-ff		# or "local-v1.0", etc.

This creates one commit (on the new local-v2.0 branch) containing the entire range of changes from the entire local branch.

First thing to do is configure your working repository to make this command easy to use:

git config sendemail.to "Project Developer's List <project-devs@example.com>"
git config sendemail.chainreplyto true
git config sendemail.thread true

If your local host doesn't run a mail server that can deliver to the public Internet then you'll need to add some additional settings to tell Git how to send mail. The following may guide you:

git config sendemail.smtpserver 10.0.1.128
git config sendemail.smtpport submission
git config sendemail.smtpuser USERNAME
git config sendemail.smtppass PASSWORD
git config sendemail.smtpencryption ssl

The way you managing branches locally will determine how you specify the changes to be sent by git send-email.

If you are using topic branches then you will send one set of patches for each topic branch. You may be asked to rebase your topic branch to the head of the remote master branch if you haven't been keeping your local repository up-to-date. You should do this.

There are tools like topGit which can help with managing a slew of branches, including the ability to manage dependencies between them.

If you are just keeping all your local hacks on one branch, and you want to submit everything upstream, then the easiest thing to do is to use the name of your branch, along with the name of the branch it was made from (normally master), as the "rev-list":

git send-email --compose master^..local-work

(note the use of "^" – the meaning of master alone would miss the initial commit as it reads "_since_ master")

Fill out the headers and write the body of the introductory message, and then all your changes will be on their way!

Note you can use the --dry-run option to see what will happen without actually sending any mail.

Note also that if you're working in a clone of a git svn cloned repository then you will probably have an initial commit containing all the .gitignore files created from SVN. In that case you would use master..local-work instead of master^..local-work because you do want to ignore that initial commit – the SVN maintainers upstream won't be interested in your .gitignore files!

(more needs to be written on this topic!)

Use git up to update all locally created tracking branches!

Typically one will want to create a new branch from master to work on something like a new feature to be merged at a later date, and to push this branch to the central server (origin) and then pull any changes from other developers into the local branch, thus this new branch must be a remote tracking branch.

Unfortunately there is no easy way to do this with Git directly in one command, though Magit makes it a wee bit easier.

• create a "spin off" branch: This is the only way to create and check out a new branch without first stashing any local changes.
• unset the upstream: first just unset it
• set the upstream: then set it to the upstream name of the new branch, i.e. origin/new-branch-name (if you're following these steps in order you'll still be in the branch pop-up menu, so you don't need to press b again)
• finally set the push remote too

M-x magit-status <RETURN>
b s feature-branch-name <RETURN>
b u
u origin/feature-branch-name <RETURN>
p

Use git svn to fetch and create a "clean" import of the entire SVN repository, and if you have a Git sever then do there:

mkdir project.svn-2-git
cd project.svn-2-git
git svn init -s http://svn.example.com/project
mv .git/* .
rmdir .git
git config core.bare true
git svn fetch --all
git branch -f -t master remotes/trunk

Some details:

The -s option tells git svn that the SVN repository is layed out the standard way (trunk is in trunk/, branches are in branches/, tags are in tags/). For anything non-standard, use --trunk=TheTrunk --tags=TheTags --branches=TheBranches as necessary. FreeBSD is an example of a project using non-standard layout for SVN branches and tags, indeed one with multiple branches directories, and the following might be an example of how to fetch the most commonly used ones: -b stable/ -b release/ -b releng/ -b user/ -t release/ -T head

(Note: Don't try to use the --prefix= option – it will only cause confusion and chaos.)

The git svn fetch --all step can take a very long time. Each change set is individually fetched from Subversion and converted to a Git revision. Omitting the --all option will fetch just the trunk, but except for the most basic uses that may not be sufficient, since for one you won't be able to properly make local topic branches from the actual release branches or tags that you may need to use in local production installs.

By default git svn fetch will have created a local branch named master, but that branch initially may be problematic, most importantly because if you've fetched all SVN branches then git svn will set master to track the most recently active SVN branch, which may not be the SVN trunk branch. Also, despite this being a "tracking" branch, it won't act like one due to this being a "bare" repository – you will have to manually re-point the master branch to the head of the SVN trunk after each git svn fetch.

Set up cron to do this regularly (or else remember to do it before you go do any work in any other local clone of this repository):

cd /pub/project.svn-2-git && git svn fetch --all
cd /pub/project.svn-2-git && git branch -f -t master remotes/trunk

Note that we are manually resetting the master branch in lieu of doing a pull operation because this is a "bare" repository (where a pull request cannot be done)

Important! To add (and/or reset) any new/changed SVN tags, you may have to run the tag copying procedure above again as well.

Perhaps you will wish to create a small script that does all three operations in sequence and then call that script once, periodically, from cron. You may wish to include an interlock in the script to prevent simultaneous invocations from tangling together.

Also, there's the issue of re-running git svn create-ignore…

# WARNING:  this script could fail to interlock properly if it can be
# invoked simultaneously with more than one name!  Don't do that!
# 
LOCKDIR=/tmp/$(basename $0).lock

if [ -e $LOCKDIR ] ; then
      echo "It appears there's already an instance of $0 running..." 1>&2
      exit 1
fi

if mkdir $LOCKDIR ; then
      : #got it!
else
      echo "Oops, just missed grabbing $LOCKDIR!" 1>&2
      exit 1
fi

# do the Git stuff...

rmdir $LOCKDIR
exit $?

At this point the newly created Git repository, cloned from a Subversion repository, can be treated as a remote read-only repository which can be cloned and worked on normally.

Each local developer will clone it to a working repository somewhere:

mkdir /work/$USER
cd /work/$USER

git clone --no-hardlinks /pub/project.svn-2-git project

# OR:

git clone git://git-server.local/pub/project.svn-2-git project

# and then
cd project
git branch local-work

This is now where you do all your feature & fix branch development and integration, etc.

Periodically you will want to update from your remote origin repository, just as you would from any normal upstream Git repository, pull forward your local master branch, and then either git merge or git rebase on your local topic branch(es) as necessary; and so on. You probably don't want to have any uncommitted changes in your working repository at that time.

git checkout master
git fetch -v --all
git rebase

then depending on what your local branch structure looks like you first need to switch your repository back to your local working branch with:

git checkout local-work

and then either merge the new changes on the master branch to it:

git merge master

or maybe, if you haven't published (pushed) your changes to another Git repository you can rebase the your branch onto the new head of the master branch with:

git rebase master