{"id":16418,"date":"2025-10-09T00:00:00","date_gmt":"2025-10-09T07:00:00","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/ise\/?p=16418"},"modified":"2025-10-09T03:05:52","modified_gmt":"2025-10-09T10:05:52","slug":"git-submodules-and-dependencies","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/ise\/git-submodules-and-dependencies\/","title":{"rendered":"Working with Git Submodules: Managing Dependencies Across Repositories"},"content":{"rendered":"<p>In complex software projects, it\u2019s common to split functionality across multiple repositories. One common scenario arises when your main project depends on outputs or components developed in another repository that&#8217;s also evolving in parallel. Managing this dependency effectively\u2014especially when only a specific part of the secondary repository is needed\u2014can quickly become challenging.<\/p>\n<p>Consider the following situation:<\/p>\n<p>You\u2019re developing a primary application hosted in <strong>Repository A<\/strong>. This application relies on certain outputs or modules that are being developed in <strong>Repository B<\/strong>, which is itself an active project. Instead of duplicating code or manually syncing changes between the two, it would be ideal to embed a specific part (or subset) of Repository B into Repository A. Moreover, you want this embedded portion to act as a <strong>snapshot<\/strong>\u2014able to be updated when needed, but otherwise stable and independent from day-to-day changes in Repository B.<\/p>\n<p>This is where <strong>Git submodules<\/strong> come into play. A <strong>Git submodule<\/strong> is a reference to a specific commit in another Git repository. <a href=\"https:\/\/git-scm.com\/book\/en\/v2\/Git-Tools-Submodules\">Git Submodules<\/a><\/p>\n<p>Before proceeding, you may find it useful to check out a couple of related posts from our team that explore similar challenges. For instance, one covers how to synchronize an open-source project hosted on <a href=\"https:\/\/github.com\/\">GitHub<\/a>, and a private downstream mirror <a href=\"https:\/\/azure.microsoft.com\/en-us\/products\/devops\/repos\/\">Azure Repos<\/a>. More information here: <a href=\"https:\/\/devblogs.microsoft.com\/ise\/synchronizing-multiple-remote-git-repositories\/\">Synchronizing multiple remote Git Repositories &#8211; ISE Developer Blog<\/a>. Or this post which addresses a common dilemma faced when working with shared NuGet packages across multiple solutions\u2014specifically, whether to manage them in a mono-repo or micro-repo setup. You can find more information here: <a href=\"https:\/\/devblogs.microsoft.com\/ise\/dotnet-multi-repo\/\">Developing with Multiple Repositories inside a Single Solution for .NET<\/a><\/p>\n<p>Now, back to the scenario at hand.<\/p>\n<p><strong>Fig 1 &#8211; Submodule:<\/strong>\nDemonstrates how Repository A includes a defined part of Repository B as a submodule, enabling controlled updates and modular development.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2025\/10\/Fig1_submodule.png\" alt=\"Fig1_submodule\" \/><\/p>\n<p>Fig 1 &#8211; Submodule.<\/p>\n<p>In this post, we\u2019ll explore how Git submodules can help efficiently manage such scenarios. As a bonus at the end, we\u2019ll also show how to build GitHub Actions on top of this.<\/p>\n<h3>Step-by-Step: Adding a Git Submodule at a Specific Revision<\/h3>\n<p>To include a subset of another repository into your project using Git submodules, follow the steps below:<\/p>\n<hr \/>\n<h4>1. Add the Submodule<\/h4>\n<p>From the main repository\u2014<strong>Repository A<\/strong> in Fig. 1\u2014you can add <strong>Repository B<\/strong> as a submodule. This allows you to include another Git repository inside your current one. In this example, we\u2019re adding Repository B (<code>https:\/\/github.com\/youruser\/test.git<\/code>) into Repository A (<code>https:\/\/github.com\/youruser\/ExampleGit.git<\/code>). Please note, those two repositories names are just random examples, they do not exist.<\/p>\n<p>To do this, use the following command:<\/p>\n<pre><code class=\"language-bash\">git submodule add -b &lt;branch&gt; &lt;repo-url&gt; &lt;path&gt;<\/code><\/pre>\n<ul>\n<li>Replace <code>&lt;branch&gt;<\/code> with the branch you want to track in the submodule.<\/li>\n<li>Replace <code>&lt;repo-url&gt;<\/code> with the URL of the submodule repository (e.g., <code>https:\/\/github.com\/youruser\/test.git<\/code>).<\/li>\n<li>Replace <code>&lt;path&gt;<\/code> with the folder name where the submodule should be placed in Repository A (e.g., <code>submoduletest\/test<\/code>).<\/li>\n<\/ul>\n<blockquote><p>Example:<\/p>\n<pre><code class=\"language-bash\">git submodule add -b main https:\/\/github.com\/youruser\/test.git submoduletest\/test<\/code><\/pre>\n<\/blockquote>\n<h4>2. Initialize and Update the Submodule<\/h4>\n<p>This fetches the content of the submodule and initializes any nested submodules if present:<\/p>\n<pre><code class=\"language-bash\">git submodule update --init --recursive<\/code><\/pre>\n<hr \/>\n<h4>3. Fetch the Latest Changes<\/h4>\n<p>(Optional, but helpful if the submodule is already cloned and you want the latest changes.)<\/p>\n<pre><code class=\"language-bash\">git fetch<\/code><\/pre>\n<hr \/>\n<h4>4. Check the Commit Log of the Submodule<\/h4>\n<p>Navigate into the submodule folder and view the commit history to select the specific revision you want:<\/p>\n<p>In this example it would be <code>cd submoduletest\/test<\/code><\/p>\n<pre><code class=\"language-bash\">cd path_to_the_submodule\r\ngit log --online<\/code><\/pre>\n<blockquote><p><strong>Note:<\/strong> Copy the desired commit hash.<\/p><\/blockquote>\n<hr \/>\n<h4>5. Checkout the Specific Commit<\/h4>\n<p>Use the commit hash from the previous step:<\/p>\n<pre><code class=\"language-bash\">git checkout &lt;commit-hash&gt;<\/code><\/pre>\n<hr \/>\n<h4>6. Stage and Commit the Submodule Reference<\/h4>\n<p>Move back to the root of your main repository and stage the submodule at the chosen commit:<\/p>\n<pre><code class=\"language-bash\">cd ..\r\ngit add path_to_the_submodule\r\ngit commit -m \"Add submodule at specific revision\"<\/code><\/pre>\n<hr \/>\n<h4>7. Push the Changes<\/h4>\n<p>Finally, push your main repository with the submodule reference included:<\/p>\n<pre><code class=\"language-bash\">git push<\/code><\/pre>\n<p>You can also add this as a GitHub Action and manually trigger updates<\/p>\n<ol>\n<li><strong>In the main repo<\/strong>, create a GitHub Action that:\n<ul>\n<li>Pulls the latest commit from the submodule.<\/li>\n<li>Commits the updated submodule pointer.<\/li>\n<li>Opens a pull request.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<h3>Example GitHub Action (in main repo)<\/h3>\n<p>Create the following file with this structure <code>.github\/workflows\/update-submodule_OR_ANY_OTHER_NAME.yml<\/code>:<\/p>\n<pre><code class=\"language-bash\">main-repo\/\r\n\u251c\u2500\u2500 .github\r\n\u2514\u2500\u2500\u2500\u2500 workflows\/<\/code><\/pre>\n<p>References:<\/p>\n<ul>\n<li><a href=\"https:\/\/docs.github.com\/en\/actions\/about-github-actions\/understanding-github-actions\">Understanding Github Actions<\/a><\/li>\n<li><a href=\"https:\/\/docs.github.com\/en\/actions\/writing-workflows\/quickstart\">Github Workflows Quickstart<\/a><\/li>\n<\/ul>\n<p>Using our scenario example:<\/p>\n<pre><code class=\"language-yml\">name: Manual Submodule Update_v2 (branch)\r\n\r\non:\r\n  workflow_dispatch:  \r\n\r\npermissions:\r\n  contents: write\r\n\r\njobs:\r\n  update-submodule:\r\n    runs-on: ubuntu-latest\r\n\r\nsteps:\r\n  - name: Checkout main repo with submodules\r\n    uses: actions\/checkout@v4\r\n    with:\r\n      submodules: recursive\r\n      token: ${{ secrets.GITHUB_TOKEN }}\r\n\r\n  - name: Update submodule 'test' to latest \r\n    run: |\r\n      cd submoduletest\/test\r\n      git checkout main\r\n      git pull origin main\r\n      cd ..\/..\r\n \u00a0\u00a0\u00a0\u00a0\u00a0git config user.name \"github-actions[bot]\"\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0git config user.email \"github-actions[bot]@users.noreply.github.com\"\r\n      git add submoduletest\/test\r\n      git commit -m \"Manually triggered: update submodule 'test' to latest\" || echo \"No changes to commit\"\r\n      git push origin HEAD:main    \r\n<\/code><\/pre>\n<p>From GitHub Actions, you should be able to see the workflow and run it, as shown in Fig 2:<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2025\/10\/Fig2_Workflow.png\" alt=\"Fig2_Workflow\" \/><\/p>\n<p>Fig 2 &#8211; Workflow.<\/p>\n<p>Inside of the Submodule folder if everything worked, it should show something similar to the Fig 3 &#8211; Results:<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2025\/10\/Fig3_WorkflowResults.png\" alt=\"Fig3_WorkflowResults\" \/><\/p>\n<p>Fig 3 &#8211; Results.<\/p>\n<h3>What This Workflow Does<\/h3>\n<ul>\n<li>Manually triggered from GitHub UI<\/li>\n<li>Goes into your submodule path<\/li>\n<li>Checks out the <code>main<\/code> branch and pulls the latest<\/li>\n<li>Commits the updated submodule reference<\/li>\n<li>Pushes the commit to the <code>main<\/code> branch of your <strong>main repo<\/strong><\/li>\n<li>It uses GITHUB_TOKEN which is a <strong>built-in GitHub Actions token<\/strong> automatically generated for each workflow run. More details here: <a href=\"https:\/\/docs.github.com\/en\/actions\/security-for-github-actions\/security-guides\/automatic-token-authentication\">GITHUB_TOKEN<\/a><\/li>\n<\/ul>\n<h4>Token Permissions &#8211; GITHUB_TOKEN<\/h4>\n<p>GitHub allows you to configure GITHUB_TOKEN permissions at the repository level. To ensure your workflow runs properly, confirm that GITHUB_TOKEN has the required permissions:<\/p>\n<ul>\n<li>Navigate to your repository on GitHub.<\/li>\n<li>Click on the Settings tab (top-right).<\/li>\n<li>In the left sidebar, go to Actions \u2192 General.<\/li>\n<li>Scroll to Workflow permissions.<\/li>\n<\/ul>\n<p>Ensure the following option is selected:<\/p>\n<p>\u2705 Read and write permissions<\/p>\n<h4>Alternative: Using a Personal Access Token (PAT)<\/h4>\n<p>If you prefer or need to use a Personal Access Token (PAT) instead of GITHUB_TOKEN, follow these steps:<\/p>\n<ul>\n<li>Go to your GitHub account \u2192 Settings &gt; Developer settings &gt; Personal access tokens.<\/li>\n<li>Create a classic PAT with the repo and workflow scopes.<\/li>\n<li>In your repository, go to: Settings \u2192 Secrets and variables \u2192 Actions \u2192 New repository secret\nAdd your PAT and name it, e.g. TOKEN_NAME.<\/li>\n<li>Update your workflow to use the PAT:<\/li>\n<\/ul>\n<pre><code class=\"language-yml\">token: ${{ secrets.TOKEN_NAME }}<\/code><\/pre>\n<h3>Alternative Action Configuration with a recursive Solution to the Workflow.<\/h3>\n<p>When using Git submodules in CI workflows, you might face issues related to branches or commits. If your <code>.gitmodules<\/code> is configured like this example:<\/p>\n<pre><code class=\"language-ini\">[submodule \"path_to_the_submodule\"]\r\n    path =  path_to_the_submodule\r\n    url = https:\/\/github.com\/youruser\/somegitreponame.git\r\n    branch = somebranchname<\/code><\/pre>\n<p>Instead of using the action setup configuration proposed earlier for the yml, you could use the following:<\/p>\n<pre><code class=\"language-yml\">- name: Pull &amp; update submodules recursively\r\n\r\n  run: |\r\n    git submodule update --init --recursive\r\n    git submodule update --remote --recursive<\/code><\/pre>\n<p>\u26a0\ufe0f This works well only if the recorded commit exists and the remote branch is available.<\/p>\n<h4>Common Error<\/h4>\n<p>If you are experiencing this issue:\n&#8220;fatal: Unable to find refs\/remotes\/origin\/ in submodule path &#8216;path_to_the_submodule'&#8221;<\/p>\n<p><code>Possible Cause<\/code><\/p>\n<p>This error typically occurs when:<\/p>\n<ul>\n<li>The submodule reference in the main repo points to a non-existent commit or branch in the remote.<\/li>\n<li>The CI environment hasn\u2019t fetched the required data.<\/li>\n<li>Also, confirm if the GITHUB_TOKEN or PAT has access to the submodule.<\/li>\n<\/ul>\n<p>Submodules are pinned to specific commits. They do not track branches by default and won\u2019t auto-update unless explicitly told.<\/p>\n<p><code>Solution Checklist<\/code><\/p>\n<ul>\n<li>Confirm the submodule and branch exist in the remote repository.<\/li>\n<li>Confirm permissions.<\/li>\n<li>Use fetch-depth: 0 in the actions\/checkout step to fetch all commit history:<\/li>\n<\/ul>\n<pre><code class=\"language-yml\">- uses: actions\/checkout@v4\r\n  with:\r\n    fetch-depth: 0<\/code><\/pre>\n<ul>\n<li>Explicitly check out submodule branches using:<\/li>\n<\/ul>\n<pre><code class=\"language-bash\">git submodule foreach 'git checkout somebranchname &amp;&amp; git pull'<\/code><\/pre>\n<ul>\n<li>Ensure your token (GITHUB_TOKEN or PAT) has access to both the main and submodule repositories.<\/li>\n<\/ul>\n<p>Full Example adapted to the fix proposed. Note this example is configured to fetch from a specific branch(somebranchname):<\/p>\n<pre><code class=\"language-yml\">name: Manual Submodule Updatev2 (branch)\r\n\r\non:\r\n  workflow_dispatch:  \r\n\r\npermissions:\r\n  contents: write\r\n\r\njobs:\r\n  update-submodule:\r\n    runs-on: ubuntu-latest\r\n\r\nsteps:\r\n  - name: Checkout main repo with submodules\r\n    uses: actions\/checkout@v4\r\n    with:\r\n      submodules: recursive\r\n      fetch-depth: 0\r\n      token: ${{ secrets.GITHUB_TOKEN }}\r\n\r\n\u00a0\u00a0\u00a0- name: Ensure submodule uses correct branch\r\n\u00a0\u00a0\u00a0\u00a0\u00a0run: |\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0git submodule update --init --recursive\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0git submodule foreach 'git fetch origin somebranchname &amp;&amp; git checkout somebranchname &amp;&amp; git pull origin somebranchname'\r\n\r\n\u00a0\u00a0\u00a0- name: Commit and push updated submodule reference\r\n\u00a0\u00a0\u00a0\u00a0\u00a0run: |\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0git config user.name \"github-actions[bot]\"\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0git config user.email \"github-actions[bot]@users.noreply.github.com\"\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 git add  path_to_the_submodule\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 git commit -m \"Update submodule to latest commit on 'somebranchname' branch\" || echo \"No changes to commit\"\r\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 git push origin HEAD:main    \r\n<\/code><\/pre>\n<h3>Summary<\/h3>\n<p>In projects where multiple repositories are involved, it&#8217;s common to need a subset of one repository (e.g., Repository B) included within another (Repository A). Git submodules provide an efficient way to manage this relationship by embedding one repository inside another while maintaining independent development.<\/p>\n<p>This post covers:<\/p>\n<ul>\n<li><strong>Scenario Overview<\/strong>: How Repository A depends on outputs from Repository B, and the need for a snapshot of Repository B inside A.<\/li>\n<li><strong>Introduction to Git Submodules<\/strong>: Why and when to use them.<\/li>\n<li><strong>Step-by-Step Guide<\/strong>:\n<ol>\n<li>Adding a submodule with a specific branch.<\/li>\n<li>Initializing and updating the submodule.<\/li>\n<li>Fetching the latest changes.<\/li>\n<li>Checking out a specific commit.<\/li>\n<li>Committing and pushing the submodule reference.<\/li>\n<\/ol>\n<\/li>\n<\/ul>\n<p>Using submodules allows for modular development, controlled updates, and cleaner project organization without duplicating code or tightly coupling repositories.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Working with Git Submodules and creating actions.<\/p>\n","protected":false},"author":192662,"featured_media":16419,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1,3451],"tags":[3317,187,3615],"class_list":["post-16418","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cse","category-ise","tag-git","tag-github","tag-submodule"],"acf":[],"blog_post_summary":"<p>Working with Git Submodules and creating actions.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/posts\/16418","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/users\/192662"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/comments?post=16418"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/posts\/16418\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/media\/16419"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/media?parent=16418"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/categories?post=16418"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/tags?post=16418"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}