StiltSoft: Using Submodules in Git
In this post we are going to tell you how we solved two problems of storing source codes for one of our projects. Initially, we had a repository with code in Subversion. We were thinking about switching to Git and splitting it into two separate repositories in order to close public access to one of them. We will show you how to migrate from Subversion to Git and splitting a Git repo into two without making a mess of the file structure for a user who has access to both parts.
Step 1: Getting data from Subversion
First, we need a file with the commiters data. It looks like this:
mkuzmich = Maxim Kuzmich <email@example.com> alexkuznetsov = Alexander Kuznetsov <firstname.lastname@example.org> rkirilenko = Roman Kirilenko <email@example.com>
Let’s save it as users.txt
Now, we check out the repository to a temporary folder (in case something goes wrong we’ll have a copy of our repository):
git svn clone --stdlayout --no-metadata -A users.txt --username svn_username https://some_svn_url dest_dir-tmp
Updating data is a good idea, since someone could commit to a repository while we were checking it out:
cd dest_dir-tmp git svn fetch
Step 2: Migrating data to Git and splitting the repository
First of all, we need a clean clone of the data we got from Subversion:
git clone dest_dir-tmp dest_dir
Then we unlink our repository from Subversion:
cd ./dest_dir cp -Rf .git/refs/remotes/* .git/refs/heads/ rm -Rf .git/refs/remotes
Putting the private part to a repository
To do that, we move the private data to another folder and make a repository of this folder:
git init git add . git commit -m "initial commit"
Then we copy the private data to the server:
git remote add stash http://192.168.0.234:7990/scm/proj/secure.git git push --all stash
And now it’s time to set access rights for the private part (we used Stash for that):
Removing info about the private part
To prevent private data from getting to the sever that contains the public part, we need to remove Git control over them. Then we can copy the repository to the server:
git rm secure -r git commit -m "removed secure part" git remote add stash http://192.168.0.234:7990/scm/proj/repo.git git remote rm origin git push --all stash
Integrating the private part into the public part as a submodule
Now we add the private part as a submodule:
git submodule add http://192.168.0.234:7990/scm/proj/secure.git git commit -m "added secure part as submodule" git push stash
Step 3: Getting the whole repository (with the private part).
We cloned the repository:
git clone http://192.168.0.234:7990/scm/proj/repo.git
As a result we got an empty folder named secure, there’re no data in it and rights are not checked when accessing the submodule’s content. Therefore everyone can work with the repository. To get access to the private part, we need to initialize the submodule and get the content of the public repository:
git submodule init git submodule update
At this stage, user rights will be checked and if everything’s ok, the user will get the private part from the public repository.
When working with submodules, please note that a submodule stores a link to a specific commit of the public repository. In case the data in the privaterepository have changed and the changes should be merged with the main repository, you’ll need to update the submodule data and commit them to the main repository.
git submodule foreach git pull origin master git add . git commit -m "updated submodule" git push origin
However it may look like a difficult task, migrating a small repository from Subversion to Git only takes around 15 minutes. You will need another 15 minutes for splitting the repository and setting access rights for the private part. You may want to use Atlassian Stash for making your work with Git more intuitive and user-friendly.