Git: merge two repositories one over the other

Question

I have a repository A (the original one) having its full set of commits. At a point in time it was decided that repository A wouldn't be used anymore and a new clean repository B was created from scratch putting all repo A's contents using copy + paste (not by merging A's contents into the clean B, so that to keep commits history) in it. What I want to achieve now is to glue both repositories' commits and history (it is very important that history is glued) in a third new clean repository C. The final aim is that repository C should contain all changes from both source repos A and B in chronological order as if repository B wasn't created at all e.g. as if work has proceeded in repository A.

So the current situation is:

repo A: commit A1 > commit A2 > commit A3 > ... > commit An

repo B: commit B1 (= A1 + A2 + A3 + ... + An) > commit B2 (= commit An+1) > ... > commit Bk

Tried the following approaches:

git remote add -f A <repo_A_url>
git merge A/master
git remote add -f B <repo_B_url>
git merge B/master

Then resolved conflicts. Partially worked since history was messed up somehow.

Then tried I think the cleanest approach - merge only repo A into repo C and then cherry pick repo B's commits ranging from B2 ... Bk. It works but cherry picking merge commits slows down the merge process.

I could be duplicating a topic and please tell me if I do, since crawled through many threads and mostly seen how to add two repos to a third one by positioning them in different folders in the final repo which is not the case. If you could share the most suitable, sematically correct and git-friendly approach possible.

Big thanks.

Answer 1

You can cherry pick a range of commit by a single commit.

$ git merge A/master          # merge A/master into C/master
$ git cherry-pick B2^..Bk     # cherry-pick all B2 to Bk commits

Or, you can rebase. Go to C repo master branch. You can checkout a new banch (say, rebase) then Merge B/master and rebase it onto A/master

$ git checkout -b rebase origin/master # checkout a new branch with clean C-remote/master history 

$ git merge B/master            # merge B/master into C/master
$ git rebase A/master           # rebase C/master onto A/master 

If all histories looks ok then replace master by rebase branch.

$ git checkout rebase          # make sure current branch is rebase 
$ git branch -D master         # delete local master branch

$ git checkout -b master       # create & checkout a new 'master' branch      
$ git push -f orgin master     # update C-remote/master

Answer 2

If your history is pretty linear (i.e. just master in each of A and B), this won't be too bad. If there are a lot of branches to contend with it will get more involved (see update below), but still this would provide a starting point:

First, create your C repo.

git clone /url/for/repo/A C
cd C

Now fetch all the objects from B

git remote add B /url/for/repo/B
git fetch B

Now we should have both histories in the new C repo. Rebase the B repo commits onto the A history

git rebase --onto master --root B/master

Now you need to update the master ref and maybe clean up a bit

git branch -f master
git remote remove B

Now A is listed as your origin remote; you probably either want to push to it or remove it as an origin, depending on whether you intend for A to contain everything going forward.

• UPDATE: Ok, here's a little more info in case your history isn't linear, since it probably never is... First let's focus on topology, then a word or two on refs...

Topology-wise, there are really three overall scenarios:

1) Branches and merges in B

So let's suppose you have a single branch tip in B (if not, read through this but then see scenario 2), which traces back to a single root you can graft onto a single branch tip in A (if not see scenario 3 below then come back to this).

A1 ---- A2 ---- A3  <--- (master)
   \          /
     A4 -- A5 

---------------------------

B1 ---- B2 ---- B3 <--- (master)
   \          /
     B4 -- B5

There might have been branches in A, but ultimately they've been merged back and you don't have to worry about them. There also might have been branches in B, and though they're merged back they could cause trouble if rebase tries to make them linear.

Start by creating C and importing both histories, as we did above. (In the end I'll suggest a slight variation to this procedure, having to do with refs... but let's come back to that.)

Now, you have two options here. The simplest by far is to use filter-branch (but it may be time consuming). Find the hash value of the commit onto which you will be grafting (marked A3 above) and run

git filter-branch --parent-filter 'sed "s/^\$/-p xxxxxxxxxx/"' B/master

(where the xxxxxxxxxx is the hash value).

That does assume you have sed; it's available in the git bash environment on Windows, or on pretty much any *nix system. If not, you can come up with an equivalent filter. (All it's doing is saying "if the input is an empty line, write out '-p' followed by the hash to which I'm grafting; otherwise pass the input through as my output".)

If for some reason you can't do that, or if it does seem to be a performance problem, then you could try plan b: giving the --preserve-merges option to rebase... and this will do what you want a lot of the time. But there are major caveats.

Basically if a merge introduced manual changes - either because manual conflict resolution was needed, or because the merge was done with --no-commit and manual changes were introduced in that way - then the merge won't be reproduced properly by rebase even with this option.

In the case of a conflict, the rebase should stop and let you re-apply manual resolutions (which you might be able to do with a path checkout (checkout ... -- .) of the original merge commit. But in the case someone used --no-commit rebase won't even realize anything is wrong.

If you know this will be a problem but can identify one or two problem merges, then one option is to rebase each of the problem merge's parents, then redo the merge manually, then continue from that point onward.

If you don't know if/where problems will occur, you could try the rebase and then run a validation to compare commits. Before doing the rebase

git checkout master
git tag old-B-master

Then try the rebase

git rebase --preserve-branches --onto master --root B/master
git tag new-B-master

Then do whatever level of validation seems safe to you. (Obviously diff old-B-master against new-B-master at a bare minimum. When I did something like this, I wrote a script to recursively traverse the commit ancestries comparing commit by commit. Paranoid? Maybe.)

Unless this goes very, very smoothly, you're likely better off falling back to the filter-branch approach.

2) Multiple branch tips in B

A1 ---- A2 ---- A3  <--- (master)
   \          /
     A4 -- A5 

---------------------------

B1 ---- B2 <--- (master)
   \
     B4 -- B5 <--- (branch1)

Maybe your B repo isn't fully merged. This may or may not complicate things. If you're using filter-branch, it can work on many refs at once. You probably can't just say --all (because that could catch refs that are already in the A tree and likely the operation would end up failing), but you can list the branch tips from the B tree.

git filter-branch --parent-filter 'sed "s/^\$/-p xxxxxxxxxx/"' B/master B/branch1

If you're trying to use rebase (or if you just want to work with a single tip ref), you can create a temporary octopus merge.

git checkout B/master
git checkout -b b-entry-point
git merge -s ours B/branch1 B/branch2 ...

The resulting merge commit is temporary. (You can delete the b-entry-point branch after the graft.) It just provides a single "entry point" into the B commit tree.

3) Multiple branch tips in A

A --- Am <-- (master)
  \
    Ab1 <-- (branch1)
---------------------------------------
B1 ---- B2  <-- (master)

B3 ---- B4  <-- (branch1)

So what if A was not fully merged in the first place? When you created repo B, did you only create a single new commit B === Am? I'm guessing so, because you'd have had to do something weird like multiple history trees to include a representation of Ab1, and you'd later have had a little headache if you wanted to re-merge...

If you do have multiple trees to graft, then I think you'd just have to handle each one separately. Not much to do that would improve that.

If you have multiple graft points but they've since been re-merged at M, then you'll likely have to graft each of M's parents individually, then recreate the merge as M', then continue grafting the children of M onto M'.

Ok, but what about the refs?

Now the above is fine, but you may have refs (in A and/or in B) that you care about other than just the master branch.

This is one of those things that's handled better by filter-branch; in fact if I recall correctly rebase won't rewrite any ref except the branch tip its rebasing from (and not even that if its a remote branch ref).

Especially if using filter-branch, you might find it convenient to create C by cloning B and importing remote refs form A (rather than the other way around, as shown above) so that you can have filter-branch rewrite the local refs for you.

Even so, you may find some combination of remote refs that you need to relocate. The branch and tag commands with the -f option can be used as needed, aligning the local refs to whichever remote ref is most appropriate for your end state.