.NET Microservices: Project Structure with Git Submodules
After working in microservice architecture for many years I have concluded there are many different strategies to project structures. In some cases, these solutions share libraries, and code, and may rely on one another as a whole solution. With the advancement of .NET Aspire, I think this structure for a project solution is very useful. It takes into consideration a lot of different approaches and still offers an enormous amount of flexibility.
With anything related to software, there will be trade-offs. While microservice architecture is advanced so is this approach to repository management. We should always consider where the team is what they are capable of doing, what editors we are using, and what’s more productive at that current moment.
- A “root” parent Git repo with submodules.
- Atomicity: Individual Git Repositories for each Project
- Easily see build issues, step through code
- Very good support in Visual Studio
- Can be used to troubleshoot complex projects where code has been shared.
- Greater flexibility in branching strategies.
- Docker build contexts are relative (multiple Dockerfiles)
- More complex and not for beginners
- In other editors, situations there may be less support for Git Submodules and this requires a higher level of Git command line knowledge.
- It can be confusing, especially for developers who don’t understand Git well.
- Using re-useable libraries (NuGet packages) can be extremely risky and create a lot of unnecessary dependencies leading to a DLL nightmare. (Some developers choose or inherit code like this… here’s a way to deal with it…)
The root project will pull all of the sub-projects in through Git Submodules. This is very useful because a solution can contain many sub-projects and connect everything. In some cases, this would allow easy step-through of code with libraries (NuGet packages).
Microsoft keeps improving .NET and making containerization easier and with the preview release of .NET Aspire we can get a good idea of where they are going with this. I believe this stemmed from Project Ty, but this cloud-ready stack pulls Application Logs, Containers, Metrics, and Configuration into a single dashboard known as “.NET Aspire“. There is also the ability to do orchestration of adding services like a Redis Cache.
Submodules: Individual Projects
By using Git Submodules we can add individual projects to the Root Project. This will allow the autonomy of those repositories while having a single solution for developing a solution that may use multiple microservices.
Shared Libraries (DLLs/NuGet Packages)
Sharing a library between multiple projects can bring some cohesion to a solution. It’s very controversial and heavily debated among developers. This is something I will harp on here to provide my experience and perspective on the different approaches.
There are a lot of different risks and challenges to sharing code among microservices.
NuGet Libraries & Build Pipelines
I’m not crazy about creating shared libraries in the form of NuGet packages. This sounds wonderful, but in practice, there are a lot of negative tradeoffs, risks, and concerns that I have with this choice. I find them very difficult to troubleshoot and support. Think about if we have a NuGet package that is referenced by many projects and a build pipeline then we have to approve, merge code, run the build pipeline, and update the projects just to validate the changes in the package, what a headache.
I prefer the submodule approach here, I can switch out my references from NuGet packages to the actual library and step through or, I could clone that project locally and add it as a referenced project. The submodule approach is more fluent with the team by keeping this process consistent and everything all in one place with one structure.
Shared Libraries in a Single Repository
With this approach, it is very common for developers to have a single repository with multiple projects using a shared library. This works fairly well, however, this approach doesn’t allow as much granulation, control, and independence as using git submodules. This means that if we are making changes in the library we are also making changes in other projects and everything would have to be merged at once. I find this more practical than using NuGet packages, it’s far less cumbersome.
If we choose to do this then we should make smaller changes as opposed to larger sweeping changes. That way merges go smoother.
Referenced 3rd Party Libraries Blocking Updates
This is probably the biggest “gotcha” with sharing code whether it is through a referenced project or a NuGet package, but often, developers will reference and implement code into their libraries that can’t be managed easily. What I mean by this, is I’ve seen solutions where a library using AutoMapper references a version that isn’t compatible with a higher version of .NET. This meant that it wasn’t possible to update the microservice without updating the package, but, we weren’t able to update the package without updating all of the other microservices. This results in a drift between the microservices and packages. It updates everything or not… and this doesn’t allow for the independence we need in a microservice architecture.
I would personally copy as much code as I can into the individual microservice. Most of it’s boilerplate-like code that should be specific to that service.
Team & Cross Team Risks
I’m not opposed to sharing libraries, it just all depends on the circumstances including how large the project is, how much control the team has over the microservices, etc. If an entire team has domain and control over the entire project it seems to go, okay, however, when two separate teams try and do this it can be catastrophic. There is always a lot of miscommunication or no communication at all. Certain types of code should not be put in a NuGet package because microservices should have atomicity and exist without any other outside dependencies. If we share a library and pin-point a version of AutoMapper then all of the projects that inherit it are dependent upon that version of AutoMapper. This can make upgrading projects very difficult and potentially distribute vulnerabilities to projects.
Do this as little as possible. If necessary be extremely conservative with this choice. Other teams may have different testing processes and go through a completely different release process. These kinds of changes can impact others heavily.
This is a big “no-no” in microservice architecture but it’s also a very common problem. I would argue that each microservice should have its own models. That way, when we update a library we aren’t breaking internal business logic that the app may rely on or impacting the design of the microservice’s database.
If we were to create a library to share data, especially when dealing with event bus-driven architectures, I think this can be done well but several things have to be taken into consideration. There are a couple of tricks here to make this work, use Abstract Models, Model Interfaces, and/or combine this with AutoMapper to map to the microservice’s models. That way, if something changes, it’s easy to update and manage.
Visual Studio / Git CLI Considerations
I find that the vast majority of developers, especially the ones who came from Team Foundation Server (TFS) traditionally use the interface to push and pull their code. This may be out of habit, but, not all .NET Developers will know how to use Git via command line. My opinion here is that it will be easy for developers who are strong with Git CLI to set this up and developers who are not can easily push and pull their changes via Visual Studio.
Visual Studio has Amazing Support for Git Submodules
I love how incredible Visual Studio is and it has incredible support for Git Submodules. It’s very easy to select which repository we want to push, pull, and merge code with.
This is another challenging aspect of doing projects like this. While each Project contains project metadata that allows for defining where the build context is, this gets tricky when we reference files outside of the individual project.
Docker Build/Run Scripts
I often create PowerShell scripts that build and run the Dockerfiles. I prefer them to be in the root project, with a Git Submodule approach this is much more challenging. The reason being is that the Docker build file context is relative to the path. This can be impacted in multiple ways, if we want to copy in our local versions of NuGet packages or Shared Libraries we will have to create a separate
Dockerfiles.root for those scenarios.
Tutorial: Creating a Root Project
First, you’ll want to create the root repository that will pull in all of the Git Submodules. Once this project is created we will then add submodules to that project.
Adding a Git Submodule
There are several different ways to add Git Submodules but this covers the important concepts. The
-b main specifies which branch to add to the repository. The trailing
code/Common specifies the folder that it will go into. This is important for easily managing the project.
Adding the Common Shared Project (.DLL)
Add the common repo as a submodule with the main branch
<span style="font-family: inherit; font-size: inherit; color: initial;">git submodule add -b main </span>firstname.lastname@example.org:mrjamiebowman-blog/microservices-projectstructure-common.git code/Common
# update your submodules
git submodule update --remote
Adding the API Project
Add the “API” repo as a submodule with the “main” branch
<span style="font-family: inherit; font-size: inherit; color: initial;">git submodule add -b main </span>email@example.com:mrjamiebowman-blog/microservices-projectstructure-api.git code/Api
# update your submodules
git submodule update --remote
Adding the Web Project
Add the “Web” repo as a submodule with the “main” branch
<span style="font-family: inherit; font-size: inherit; color: initial;">git submodule add -b main </span>firstname.lastname@example.org:mrjamiebowman-blog/microservices-projectstructure-web.git code/Web
# update your submodules
git submodule update --remote
Once the Git Submodules are added we’ll see references to them in the repo that will contain a shortened
SHA-1 hash ID to their current state in a repository.
Tutorial: Cloning a Root Project
Cloning a repository with Git Submodules can be done in several ways. We can either do a recursive clone or clone the repository and then update.
git clone --recurse-submodules email@example.com:mrjamiebowman-blog/microservices-projectstructure-root.git
Clone & Update
git clone firstname.lastname@example.org:mrjamiebowman-blog/microservices-projectstructure-root.git
# initialize submodules
git submodule update --init --recursive
# update all submodules
git submodule update --recursive --remote