Why Resilio & Jenkins
Key in reliably delivering software builds and artifacts is a solution that automatically overcomes all possible errors without human intervention. Most systems are prone to failure. You should be certain that your synchronization set-up for CI/CD is designed for high reliability.
Resilio Platform enables fast and continuous delivery. It combines peer-to-peer, WAN optimization, difference deduplication, real-time folder monitoring, and compression into a single solution, which makes asset and artifact distribution very fast and highly reliable.
Connect will keep your jobs running even in case of an IP address change, lack of storage space, reboot and other common obstacles to smooth distribution. Connect also offers granular control over distribution timing and bandwidth consumption per endpoint. Most importantly for an automated CI/CD environment, Resilio can invoke scripts and applications at any point in the distribution cycle.
The cost of downtime and distribution delays is high. Combining Jenkins with Resilio Platform makes critical CI/CD workflows more resilient and faster.
-
Transferring artifacts (builds, images, assets) between offices
Imagine the build machine is in the cloud or in one of your offices, and you need to deliver artifacts to several offices across the globe. The main issues are usually transfer speed over wide area network (WAN), and the ability to prioritize or limit the bandwidth used by the transfer. Offices can usually download faster than the central office can seed the data. So, ideally, you would have a way to leverage other offices’ bandwidth to deliver the builds faster. You also might need specific rules regarding the distribution to each office, as well as independent policies on build retention and deletion. Connect accelerates and makes more reliable these kinds of transfers.
-
Transferring builds within an office
Imagine you need to distribute builds within the office. While the speed of delivery over the LAN is fast your central server may slow down or become overloaded with concurrent requests from many machines. Sometimes the ability to push builds to machines automatically upon completion can save a lot of time for team members. You want to give users the ability to choose the build they want, and download the build from a local server, remote server or the cloud. Connect enables this and makes it possible to have non-stop operations for CI/CD.
-
Selective download of artifacts by Dev and QA
Sometimes you need a system to give specific build access to developers and QA as a part of CI/CD process. You want team members to see all the builds, choose the specific build that is interesting to them, and download it. With Connect your sync software can detect the build location, whether a peer, local server, remote server or cloud, and provide the team member with the best source automatically. This eliminates the need of specific users to choose between build locations.
-
Push builds to virtual or physical machines globally
In most automated test environments, the download isn’t initiated by a user. Instead, the system needs to push a build to machines and trigger automated testing. Maximum reliability is essential in fully automated deployments and Connect provides that for a resilient CI/CD. In addition, Resilio Platform can also trigger the installation and start the test upon file delivery.
An Automated Continuous Distribution Example
We’re going to show you how to use Connect and Jenkins together to build an automated distribution workflow.
In the example you will distribute an executable from a source machine to a group of destination machines. This simple Resilio Platform job will:
- Take a build (hello.exe) from the source machine
- Distribute it to a group of destination machines
- Start the binary of hello.exe when it is delivered to a machine
The following examples assume you have Resilio Platform installed in each development machine, including any local or cloud virtual machines that have Jenkins installed and have access to a command line interpreter (shell) on the source machine. For more info visit https://www.resilio.com/connect/ and https://jenkins.io/
How to Configure Resilio Platform Distribution Job for Jenkins
1) Install and start Resilio Platform Management Console. Then install one agent on the build server (this agent will be the source) and other agents, on one or more different endpoints, as the destinations for artifacts.
2) Now let’s create a new Distribution job to allow Resilio Platform to take build artifacts from the desired folder on our build server and distribute it to our destination agents.
3) First, choose Distribution as a job type. Then, on step 2, specify job name and description (optional). Click Next to proceed to the next step.
4) Let’s choose “Builder” as our Source agent and specify the path to the build-artifacts directory.
5) In this step we will choose our destination agents (for simplicity, we will group them). To do so, let’s create a new group and name it “Hello world build delivery DST”, then add the destination agents to this group. Now, we’ll save this group and select it as our Destination. In this example, we leave the path as is (this is the path to the directory that will store the files). Click Next to move to the next step.
6) On this step we will specify a trigger that will run the build binary (hello.exe) on all destination machines, as soon as they complete downloading of the build.
7) On the Jon Scheduler step we’ll specify that this job will run manually (later on we’ll configure that this job will be initiated by Jenkins). When done click Next. On the next step we will leave the defaults as is, and go ahead to review the job summary and save it.
8) Now, since we want Jenkins to initiate the job we have just configured, we will create an API token which will allow Jenkins to communicate with Resilio Platform Management Console. To do so, we will configure a new API token on the API Administration page.
9) On the API Administration page click on Create New Token button and specify a name for the token, then click the Save button to save.
10) You should now see the created token. In the next steps we’ll cover how we use this token while configuring the Jenkins build workflow, so this distribution job could be initiated by Jenkins.
Execute the Resilio Platform – Jenkins Distribution Job
We will use the Resilio Platform API to execute the Distribution Job we created earlier. Resilio Platform API is RESTfull, so we will use a curl to generate the API call to trigger the distribution job.
curl --insecure --header "Authorization: Token RESILIO_MC_TOKEN" --header "Content-Type: application/json" \
-XPOST https://127.0.0.1/api/v2/runs -d "{\"JOB_ID\": "1"}"
Where JOB_ID in our case is ‘1’ since this is the first job we created. The RESILIO_MC_TOKEN is the token we generated in step 12. Use your Resilio Management Console IP address instead of 127.0.0.1
You have a couple of options to trigger this job from inside Jenkins CI:
1) Specify this step as the last step of your build script
Save it and it’s ready to be run. RESILIO_MC_TOKEN env var must be injected into the build process in Jenkins in this case.
Or,
2) Create additional distribution step in Jenkins workflow and specify it as a post build step in your workflow.
Now, all that is left is to start a build process in Jenkins and Connect will seamlessly take care of the builds distribution to all of the machines you’ve specified as Destinations.
You can at any time go back to the Connect Management Console to modify the job, view extensive reports about current and past events, review logs of trigger execution and much more.
For more info, visit Resilio Connect.
