## Using Overleaf with Git Submodules Overleaf is useful for editing manuscripts on the cloud, sometimes synchronously with collaborators. However, editing synchronously in real-time with collaborators on an Overleaf project is exceedingly rare (for me, at least). One may also find compiling on Overleaf to be slower than working locally on a LaTeX editor. In any case, one may still prefer the manuscript as an Overleaf project to be embedded within a GitHub repo that contains other scripts (e.g., Python/R/Stata scripts). ### Importing an existing GitHub repo from GitHub The easiest way for relatively small and trivial projects is to use the existing Overleaf function to import an existing GitHub repo as an Overleaf project by clicking on the New Project button. Importing a GitHub repo as an Overleaf Project However, this does not always work. One of the reasons it won't work is if the Github repo (as the superproject) is large. Overleaf won't support these repos for import. GitHub repos larger than 50mb are not supported by Overleaf The alternative is to use the Overleaf project as a standalone manuscript project and as the submodule in a superproject containing all other project assets (e.g., data files). --- ### Using Git Submodules for Overleaf Manuscripts Pt. 0 First, the Overleaf manuscript project should be linked to a GitHub repo. If it's already linked, skip [forward to pt. 1](#using-git-submodules-for-overleaf-manuscripts-pt.-1). If it is not already a Github repo, link it by selecting Menu button and then the GitHub button, which gets you to the GitHub sync modal. Syncing GitHub repo The Overleaf project should now reside in a GitHub repo like *user/manuscript* on GitHub (https://github.com/user/manuscript). (* *You can obviously name it anything else.*) Now that the Overleaf manuscript is linked to a GitHub repo, we can add it as a submodule to the existing project. --- ### Using Git Submodules for Overleaf Manuscripts Pt. 1 **CD** to the remote GitHub repo folder containing some existing assets of the project (e.g., a folder with data files `data/`, a readme documentation `README.md`, and a folder with coding scripts `scripts/`). ```bash $ cd my_project $ ls data/ README.md scripts/ ``` **Add the submodule** from https://github.com/user/manuscript (If the GitHub repo does not exist yet, [refer to this](#using-git-submodules-for-overleaf-manuscripts-pt.-0)). You can see the new changes in your local repo when you check the `git status`. ```bash $ git submodule add https://github.com/user/manuscript.git $ git status On branch main Your branch is up-to-date with 'origin/main'. Changes to be committed: (use "git reset HEAD ..." to unstage) new file: .gitmodules new file: manuscript $ ls data/ manuscript/ README.md scripts/ ``` `Git submodules` added the `manuscript` submodule into the root directory of my_project. There is also a new `.gitmodules` file. This is a configuration file so that Git knows how to map from the local directory to the `manuscript` submodule on GitHub. ```bash $ cat .gitmodules [submodule "manuscript"] path = manuscript url = https://github.com/user/manuscript.git ``` **Push the submodule** to the project's Git repo. ```bash $ git commit -m "Add manuscript submodule" [main fbace23] Add manuscript submodule 2 files changed, 4 insertions(+) create mode 100644 .gitmodules create mode 160000 manuscript $ git push origin main ``` The `160000` mode just means that the commit is as a directory entry rather than a subdirector or a file. **Pull upstream changes** from the submodule remote, if changes exist. This checks for new work in the submodule and prevents future merge conflicts. Just `cd` into the submodule and `git pull` as usual. ```bash $ cd manuscript $ git pull origin main ``` To **change locally and push to submodule repo** from the local repo, just `cd` to the submodule folder and do the usual `add-commit-push` to the submodule's remote repo. (changes are sometimes are collected in [detached heads](#gotcha-with-detached-heads).) ```bash $ cd manuscript $ git add *** $ git commit -m "Some changes from local" $ git push origin main ``` Finally, we may want to **push changes in the submodule to the project repo**. That's because changes have been pushed to the `manuscript` repo but not to the overall `project` repo. This can be seen by going back to the project root directory. A `git status` will show that the changes in the submodule folder as modified but not pushed to the project repo. ```bash $ cd ../ $ ls data/ manuscript/ README.md scripts/ $ git status On branch main Your branch is up to date with 'origin/main'. Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git restore ..." to discard changes in working directory) modified: manuscript (new commits) ``` The usual `add-commit-push` will resolve this. --- ### Gotcha with detached Heads Sometimes the changes inside the `submodule` folder might be collected in a detached `HEAD`. To confirm this, do a `git branch`. ```bash $ git branch * (HEAD detached at 660da63) * main ``` So we need to make a branch, switch back to `main` (or `master`) and then merge so that the new changes are in `main`. First, make a temporary `tmp` branch for the detached head. Then checkout `main`. Merge commits from previously detached head into `main`. Delete the temporary branch and go back to the `add-commit-push`. ```bash $ git branch tmp $ git checkout main $ git merge tmp $ git branch -d tmp $ git branch * main ``` --- ### Cloning a Git repo with a submodule Start by cloning a git repo as usual. ```bash $ git clone https://github.com/user/project.git Cloning into 'project'... ... $ ls data/ manuscript/ README.md scripts/ ``` But `cd`ing into the manuscript `submodule` folder reveals that it's still empty. ```bash $ cd manuscript $ ls . ``` So we need to `init` the local config file, and then do a `git submodule update` to fetch all the assets from that project and check out the appropriate commits listed in the superproject. ```bash $ git submodule init Submodule 'manuscript' (https://github.com/user/manuscript.git) registered for path './' $ git submodule update Cloning into 'D:/project/manuscript'... Submodule path './': checked out '405998645a301ee47ab43125ec01fd2e7a48671c' $ ls figs/ ms/ tabs/ ``` --- ### Resources * [How do I connect an Overleaf project with a repo on GitHub, GitLab or BitBucket?](https://www.overleaf.com/learn/how-to/How_do_I_connect_an_Overleaf_project_with_a_repo_on_GitHub%2C_GitLab_or_BitBucket%3F) * [Git Tools - Submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules) * [What to do with commit made in a detached head](https://stackoverflow.com/questions/7124486/what-to-do-with-commit-made-in-a-detached-head?rq=1)