How to automatically deploy add-ons to JIRA, Confluence, Bitbucket using TeamCity

August 25, 2016
#Bitbucket#Integration#How To
2 min

Connect your Bitbucket to the TeamCity CI/CD server with TeamCity Integration for Bitbucket to run builds right from Bitbucket in one click, view build results, logs and artifacts right next to your code, and never lose time switching.

TeamCity from JetBrains is one of most famous systems that people use to build their own continous integration and continous delivery solutions. We at StiltSoft use TeamCity in our development process for different purposes, as follows:

  • building add-ons
  • running integration tests for checking add-on issues before their deployment to production systems
  • running the code quality analysis with SonarQube
  • deploying updates or add-ons to our test, staging or production instances

Today we’re going to tell you how to use TeamCity for automatic deployment of add-ons to Atlassian JIRAConfluence or Bitbucket.

Continue reading “How to automatically deploy add-ons to JIRA, Confluence, Bitbucket using TeamCity”

How to mirror Bitbucket Cloud and GitHub repositories to Bitbucket Server / Data Center

October 8, 2015
#Bitbucket#How To
13 min

Visualize, track and analyze repositories hosted in Bitbucket with Awesome Graphs. Benefit from what graphs, charts and reports can tell you. Learn more

When working on our add-ons, we at Stiltsoft use Bitbucket Data Center for distributed version control management and code collaboration. Some of you have also picked Bitbucket Server / Data, and others went with another solution, e.g., Bitbucket Cloud, GitHub, or are using several platforms at once. Whatever tool you work in, mirroring repositories can come in useful in some instances. That is why we posted the article that covered the benefits and how-tos about mirroring remote repositories from Bitbucket Data Center and Server. Meanwhile, creating mirrors in Bitbucket Bitbucket Server / Data Center rather than mirroring from it might be more relevant for some of our blog readers, so today’s post is about that.

Benefits

There’s a number of goals that can be attained by mirroring repositories from Bitbucket Cloud or GitHub to Bitbucket Server / Data Center. Among them:

  • protecting yourself from downtime when Bitbucket Cloud or GitHub are unavailable
  • consolidating repositories in one place
  • facilitating the process of migration from Bitbucket Cloud or GitHub to Bitbucket Server / Data Center

How to mirror Bitbucket Cloud repositories

To arrange automatic update of mirrored repositories you can:

  • Use an add-on for Bitbucket Server
  • Use a Continuous Integration tool
  • Use an OS job scheduler

In our examples, we’ll be mirroring a Bitbucket Cloud repository to Bitbucket Server / Data Center.

Use an add-on for Bitbucket Server / Data Center

ScriptRunner for Bitbucket can come in handy when you are looking to mirror some or all of your Bitbucket Cloud repositories to Bitbucket Server / Data Center.

It’s very simple.

  • Once the add-on is installed, navigate to Bitbucket Server / Data Center Administration and select Built-in Scripts in the Script Runner Section.
  • There, choose Mirror Bitbucket Team. You can mirror repositories both from team and user Bitbucket Cloud accounts.
ScriptRunner-Built-in-Scripts
  • Enter your Bitbucket Cloud Team or User, select the target project in Bitbucket Server / Data Center, provide your Bitbucket Cloud user credentials.
ScriptRunner-Mirror Bitbucket Team
  • Select one of the synchronization options (none, install hook or poll). If you choose:
    • ‘install hook’, Bitbucket Cloud will call your Bitbucket Server / Data Center instance when there are changes in repositories which will trigger synchronization
    • ‘poll’, each remote repository will be polled every 5 minutes
    • ‘none’, there will be no synchronization performed
  • In the Regex field, there is the ‘.*’ regular expression by default. If you leave it as it is, all repositories will be mirrored from your Bitbucket Cloud. To filter the set of repositories, insert a corresponding regular expression. The regular expression is done against the repository name, e.g. for the repository ‘StiltSoft Test’ you could use:
  •  
.(?=.\bStiltSoft\b.)(?=.\bTest\b.).
  • Another useful feature. If you’d like new Bitbucket Server / Data Center repositories to be created and synchronized when there are newly-added remote repositories in Bitbucket Cloud, mark the ‘Sync new’ checkbox.
  • When all set, click Run. You’ll see the table with the repositories that are being mirrored. Initially there’s the ‘Create’ label. When the process is completed, it will say ‘Exists’ instead. You may tail the application log file to track the progress.
 ScriptRunner-Mirror Bitbucket Team
  • Once local repositories are created, you can refer to the Configure Mirrored Repositories in Built-in Scripts to view and check the status of your mirrored repositories and change the synchronization type.

Use a Continuous Integration tool

If you have some Continuous Integration solution, e.g. TeamCity, in your arsenal, you can use it. Here is one of the options for how to set up automatic update of mirrors in Bitbucket Server / Data Center using TeamCity.

  • To use SSH-authentication, copy a SSH key from the TeamCity server with a build agent you’ll be using. Then add this SSH key in your Bitbucket Cloud and Bitbucket Server / Data Center account settings.
  • Then you need to create a repository in Bitbucket Server / Data Center
  • After that, create a local copy of source repository on build agent (don’t forget to add Bitbucket Server / Data Center repository as remote):
cd /home 
mkdir mirror
cd mirror
git clone git@bitbucket.org:kkolina/demonstration-repository.git
cd demonstration-repository
git remote add bitbucket ssh://git@stash2.stiltsoft.com:7999/adp/demonstration-repository.git

Now you should make some changes in build configuration in TeamCity:

  • Add a Command Line Runner build step with a script for updating mirrored repositories that will run every time this build step is invoked.

Example of the script:

cd /home/mirror/demonstration-repository 
git pull
git push --all bitbucket
git push --tags bitbucket

You may also add a VCS Trigger that will add a build to the queue when a VCS check-in is detected in the repository you are mirroring.

  • Before adding a trigger, you should attach a VSC root in the Version Control Settings:

With the URL of the original repository and Bitbucket Cloud authentication settings:

  • Now you can add a VCS Trigger:

Use an OS job scheduler

You can also use OS job schedulers (Cron, Windows Task Scheduler) and configure an external job and schedule the launch of scripts that will be pulling changes and pushing them to a mirrored repository.

  • First we clone a repository:
cd /home/mirror 
git clone git@bitbucket.org:kkolina/demonstration-repository.git
cd demonstration-repository
git remote add bitbucket ssh://git@stash2.stiltsoft.com:7999/adp/demonstration-repository.git
  • If you use Cron, you’ll need to add a command in your crontab file that will run periodically on a given schedule and will be updating mirrored repositories, e.g.:
0 0 * * * root /home/mirror/update_repos.sh 

 

This command will run from the user ‘root’ every day at midnight and trigger a script that has a number of commands to perform update of mirrored repositories.

For the case when you have one mirrored repository that should be updated, the content of ‘update_repos.sh’ will look like this :

 #!/bin/bash 

cd /home/mirror/demonstration-repository
git pull
git push --all bitbucket
git push --tags bitbucket

If you need to mirror more than one repository, include 4 commands you can see above for each repository in ‘update_repos.sh’.

How to Mirror to Remote Repositories From Bitbucket Data Center and Server

September 18, 2015
#How To#Bitbucket#Integration
11 min

Visualize, track and analyze repositories hosted in Bitbucket with Awesome Graphs. Benefit from what graphs, charts, and reports can tell you. Learn more

 

Our Stiltsoft team has extensive expertise in Bitbucket Server and Data Center. There’s been a number of posts in our blog about the app we’ve created for Bitbucket users, Awesome Graphs for Bitbucket. In case you haven’t heard about it yet, the plugin helps engineering leaders evaluate team performance, track process improvements, and deliver products faster by transforming invisible development activities into easy-to-understand insights and reports.

In today’s blog post, we’ll briefly go through some benefits from mirroring Bitbucket Server / Data Center repositories and several ways how to do it.

Benefits

Mirroring Bitbucket Data Center / Server repositories can come in very handy when you need to:

  • reduce the load on your system, e.g. when developers and Continuous Integration servers are sharing one Bitbucket instance and are constantly pushing and pulling to/from Bitbucket Server. It can cause resource issues on the Bitbucket server
  • reduce response time for Bitbucket users from different geographical locations
  • have a backup to restore data from if something goes wrong on your Bitbucket instance

How to mirror Bitbucket Data Center / Server repositories

You can use one of the following options to set up an automatic update of remote mirrored repositories:

  • Use an add-on for Bitbucket Data Center / Server
  • Use a Continuous Integration tool
  • Use an OS job scheduler

In our example, we’ll be mirroring a repository from Bitbucket Data Center / Server to Bitbucket Cloud.

Use an add-on for Bitbucket Data Center / Server

You can use Repository Mirror Plugin for Bitbucket to automatically mirror to remote repositories.

To get started, you need to enable the add-on in the ‘Hooks’ section of the repository settings in Bitbucket:

  • click the Mirror Hook text (or the pencil icon) in the Hooks section of the repository settings

mirror hook bitbucket

  • enter mirror repository URL – the GIT URL (SSH, git, http(s), file) to the remote mirrored repository
  • enter your Bitbucket Cloud user name and the password (if pushing is via HTTP(S))

mirror hook dialog bitbucket

  • click ‘Enable’

If using SSH, ensure your private key is installed for the user Bitbucket runs under. Learn how to set up an SSH key and how to troubleshoot SSH issues.

With mirror hooks, all configured remote mirrors will be receiving changes when a commit or pull request merge is performed.

Use a Continuous Integration tool

You can also use some Continuous Integration solutions, e.g. TeamCity, to pull changes from your Bitbucket server and push them to a remote repository. Below is the description of one of the ways how to do that.

To use SSH authentication, copy an SSH key from the TeamCity server with a build agent you’ll be using. Then add this SSH key in your Bitbucket Cloud and Bitbucket Data Center / Server account settings.

After that, you need to create a remote repository:

git clone ssh://git@stash2.stiltsoft.com:7999/adp/demonstration-repository.git
cd demonstration-repository
git remote add bitbucket git@bitbucket.org:kkolina/demonstration-repository.git

If you are working with TeamCity and would like to configure automatic updates of mirrored repositories, all you have to do is change the build configuration in TeamCity. You need to add a Command Line Runner build step with a script for updating mirrored repositories that will run every time this build step is invoked.

teamcity build step

Example of a script:

cd /home/mirror/demonstration-repository
git pull
git push --all bitbucket
git push --tags bitbucket

You may also add a VCS Trigger that will add a build to the queue when a VCS check-in is detected in the repository you are mirroring.

Before adding a trigger, you should attach a VSC root in the Version Control Settings:

teamcity vcs root

With the URL of the original repository and Bitbucket Data Center / Server authentication settings:

teamcity vcs root dialog

Then you can add a VCS Trigger:

teamcity

Use an OS job scheduler

As another option, it’s possible to use OS job schedulers (Cron, Windows Task Scheduler) to set up an external job and schedule the launch of scripts that will be pulling changes and pushing them to a mirrored repository.

Again, first, we clone a repository:

git clone ssh://git@stash2.stiltsoft.com:7999/adp/demonstration-repository.git
cd demonstration-repository
git remote add bitbucket git@bitbucket.org:kkolina/demonstration-repository.git

If you use Cron, you’ll need to add a command in your crontab file that will run periodically on a given schedule and will be updating mirrored repositories, e.g.:

0  0    * * *   root    /mirror/update_repos.sh

This command will run from the user ‘root’ every day at midnight and trigger a script that has a number of commands to perform the update of mirrored repositories.

For the case when you have one mirrored repository that should be updated, the content of ‘update_repos.sh’ will look like this :

#!/bin/bash

cd /home/mirror/demonstration-repository
git pull
git push --all bitbucket
git push --tags bitbucket

If you need to mirror several repositories, include 4 commands you can see above for each repository in ‘update_repos.sh’.

It was a brief overview of this topic to give you a general idea and encourage you to drill into one of these options if you believe you can benefit from it. Visualize, track and analyze repositories hosted in Bitbucket with Awesome Graphs.

Feel free to comment below.

You can also read how other teams benefit from using Bitbucket in a bundle with Awesome Graphs: