3 steps to creating local mirrored Git repositories for your Continuous tests

At IDRsolutions, we have spent the last 6 months working to make our Java code base for our Java PDF Library and PDF to HTML5 Converter more modular. This has the downside of generating lots of fetch requests across the Internet and we see timeout issues in Git which then fail our main tests. So we have reduced this by having the repositories mirrored on disk. Here is how we did it:

1. Setup Local repositories

Previously, every task would attempt to poll the main repositories. On our Hudson box, the Git mirrors now live in /Users/hudson/repositories/ and our Jenkin’s mirrors live in C:\repositories

The mirror is created by the command

git clone –mirror https://path/ToCode/module.git

The mirror tag is needed because the version you normally check out (which creates a .git directory) does not work.

Now we have a set of tasks (in both Hudson and Jenkins), which now poll the main repositories.

2. Add a task to update the mirror copy

We now add a task to listen to the original Git repository and update our mirror copy when there are changes. We do this with a simple fetch. You setup a task for each module.

sync git rep

3. Set existing tasks to use the mirror repositories

Finally you point the existing tests to use the Git mirrors on disk. You can use file::/ or just the path.

use local copiesYou will now see that the sync task will fire up to update the local mirror and then all the other tests will fire off.

As well as reducing traffic substantially, a side-effect is that the tests can now be easily disabled just by disabling the sync tasks, and you will not see your main tests fail if there is an issue with the Git pull.

We have found this makes our tests substantially more robust and we hope you find it useful too. Let us know…

If you are a first-time reader, or simply want to be notified when we post new articles and updates, you can keep up to date by social media (Twitter, Facebook and Google+) or the Blog RSS.

Related Posts:

The following two tabs change content below.

Mark Stephens

System Architect and Lead Developer at IDRSolutions
Mark Stephens has been working with Java and PDF since 1999 and has diversified into HTML5, SVG and JavaFX. He also enjoys speaking at conferences and has been a Speaker at user groups, Business of Software, Seybold and JavaOne conferences. He has a very dry sense of humor and an MA in Medieval History for which he has not yet found a practical use.
Markee174

About Mark Stephens

Mark Stephens has been working with Java and PDF since 1999 and has diversified into HTML5, SVG and JavaFX.

He also enjoys speaking at conferences and has been a Speaker at user groups, Business of Software, Seybold and JavaOne conferences. He has a very dry sense of humor and an MA in Medieval History for which he has not yet found a practical use.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>