ec2-plugin

ec2-plugin

Jenkins EC2插件实现AWS云上的弹性构建环境

EC2插件使Jenkins能够根据工作负载自动在AWS EC2或Eucalyptus上启动和终止代理实例。它支持构建集群的动态扩展、Spot实例管理和IAM安全配置。通过图形界面或Groovy脚本,用户可以灵活配置Jenkins构建环境,实现资源弹性和成本优化。插件还提供了对Windows和Unix系统的支持,以及Spot实例的配置指南。用户可以利用IAM角色进行精细的权限控制,确保AWS资源的安全访问。这些功能共同为Jenkins提供了一个高度可定制和经济高效的云端构建解决方案。

JenkinsEC2AWS插件自动化Github开源项目

ec2-plugin

Jenkins Jenkins Plugin GitHub release Gitter

Table of contents

Introduction

Allow Jenkins to start agents on EC2 or Eucalyptus on demand, and kill them as they get unused.

With this plugin, if Jenkins notices that your build cluster is overloaded, it'll start instances using the EC2 API and automatically connect them as Jenkins agents. When the load goes down, excess EC2 instances will be terminated. This set up allows you to maintain a small in-house cluster, then spill the spiky build/test loads into EC2 or another EC2 compatible cloud.

Usage

First, go to EC2 and sign up for the service. Once you've installed the plugin, you navigate to the main "Manage Jenkins" > "Configure System" page, and scroll down near the bottom to the "Cloud" section. There, you click the "Add a new cloud" button, and select the "Amazon EC2" option. This will display the UI for configuring the EC2 plugin.  Then enter the Access Key and Secret Access Key which act like a username/password (see IAM section). Because of the way EC2 works, you also need to have an RSA private key that the cloud has the other half for, to permit sshing into the instances that are started. Please use the AWS console or any other tool of your choice to generate the private key to interactively log in to EC2 instances.

Once you have put in your Access Key and Secret Access Key, select a region for the cloud (not shown in screenshot). You may define only one cloud for each region, and the regions offered in the UI will show only the regions that you don't already have clouds defined for them.

Use "Test Connection" button to verify that Jenkins can successfully talk to EC2. If you are using UEC you need to click on Advanced and fill out the endpoint details for your cluster.

Next, configure AMIs that you want to launch. For this, you need to find the AMI IDs for the OS of your choice. Packer is a good tool for doing that. Jenkins can work with any Unix AMIs. If using an Ubuntu EC2 or UEC AMI you need to fill out the rootCommandPrefix and remoteAdmin fields under advanced. Windows is currently somewhat supported.

Configuring labels allows Jenkins to pick the right AMI to start. For example, if all your existing agents labeled "solaris" are fully busy and you have more builds that are tied to the "solaris" label, Jenkins will start the AMIs that have the "solaris" label.

Init script is the shell script to be run on the newly launched EC2 instance, before Jenkins starts launching a agent agent. If the AMI doesn't have Java pre-installed, you can do this in the init script. This is also a good place to install additional packages that you need for your builds and tests.  The init script is located at /tmp/init.sh and is owned and run by the user account specified in the "Remote User" field (so use of "sudo" may be required for non-root accounts).

Configure several AMIs if you need different OS images.

With this setting, your Jenkins will automatically start instances when the load goes up, and the instances will be terminated (or stopped - see below) automatically if it's idle for more than 30 mins.

By default, instances are terminated when the idle timeout period expires. You can change this by specifying the Stop/Disconnect on Idle Timeout flat in the Advanced properties of the AMI configuration. If this is specified, the instance is stopped when the timeout expires. If the instance is required again, then the plugin will look for a stopped instance that exactly corresponds to the AMI specification and resume it if found. Otherwise a new instance is started. Having the instances be stopped instead of terminated is useful when you are using EBS volumes and want to keep them mounted for the life of the instance and reuse the instance for long periods of time. This can greatly reduce the startup time of the instance since it does not have to build the volume from the snapshot.

Spot Instances

Spot instances are similar to EC2 instances, except for a few key differences. They are generally more affordable, but they have the possibility of being terminated if someone has bid more on them than you have and can take longer to provision.  To mitigate some of these issues, your Jenkins and Agent AMIs will need a bit of configuration to convert the Spot agents to register with Jenkins when they spawn up. Due to these changes, it may appear that a Spot agent will have failed (with a red X), but the message associated with the error will inform you if the Spot instance just has not called back yet or not. For more information on Spot instances, read the information here: http://aws.amazon.com/ec2/spot-instances/ .

Enable Spot Request

To enable use of Spot as opposed to EC2 for an instance, the "Use Spot Instance" check box must be checked.  Also, a bid price must be specified.  If you want to determine what the current price of the instance is without going to the AWS website, pick your region and instance type (as you already should) and click "Check Current Spot Price".

To choose between a Persistent or One Time bid (to keep the bid alive until cancelled or to stop the bid after it has been fulfilled once), choose from the drop down menu.

Configure Jenkins for Spot Support

For Jenkins, the major configuration change will be if you have a restrictive firewall, since these instances need to connect back to Jenkins.  The first configuration change to Jenkins is to change your Jenkins URL option in the Configure Jenkins page to be the external URL to the server.  One port that needs to be open is the one that you use to access Jenkins, the other is the JNLP port, which is generally randomly assigned.  To set the JNLP port to something predictable, follow the Connection Mechanism section on this page. Jenkins CLI

Configure AMI for Spot Support

In order for your EC2 instance to know that it is to be a Jenkins agent, it must be preconfigured with start up commands so that it can register itself with Jenkins.  The Jenkins information is passed to the Spot agents via EC2 user-data.  This information includes the name that Jenkins has given the agent, and the configured URL for the Jenkins controller node.  

Sample scripts for assisting in configuring an Ubuntu-based AMI to work with the Jenkins ec2-plugin and Spot agents are included with the installation of the plugin. 
Config Script:

(jenkins_server)/plugin/ec2/AMI-Scripts/ubuntu-ami-setup.sh

Startup Script:

(jenkins_server)/plugin/ec2/AMI-Scripts/ubuntu-init.py

The config script is run by the user from the EC2 instance with root access.  It installs Java onto the instance, fetches the startup script and sets it to run when the machine starts up.  It can be retrieved from the above URL using a command like wget, or copied over using a tool like scp.

wget (jenkins_server)/plugin/ec2/AMI-Scripts/ubuntu-ami-setup.sh

Once the scripts have been downloaded, the script can be run. Running this script will also run the ubuntu-init.py script, so there is no need to run it on its own.

sudo sh ubuntu-ami-setup.sh jenkins_server{:port}

Note: "http://" will be prefixed to the jenkins_server parameter

The config script then fetches the startup script and sets up the AMI to register itself with a Jenkins controller node when it gets started.

After setting up the image, you can save the image using Amazon’s EC2 web console. To do this, right click on your instance from the console and select “Create Image (EBS AMI)”.

In order to set up additional images using other operating systems, you can configure your own startup script based on the startup script above.  This script should perform the following actions when the machine is started up:

# Verify that Java is installed # Install Java if it is not installed # Read user data for the EC2 instance. It is available from [http://169.254.169.254/latest/user-data] # Values are passed in with the format of JENKINS_URL=[Jenkins_Url]&SLAVE_NAME=[Agent_Name]&USER_DATA=[other_user_data] # Parse the values to retrieve the Jenkins_Url and Agent_Name # Fetch the agent.jar from the Jenkins controller using wget (or something similar) wget [Jenkins_Url]jnlpJars/agent.jar -O agent.jar # Register the agent to the Jenkins controller node java -jar agent.jar -jnlpUrl [Jenkins_Url]computer/ [Agent_Name] slave-agent.jnlp

IAM setup

It's possible to create a separate account for Jenkins using the Amazon IAM feature. Go to the IAM tab in the AWS console and create a user. Then go to the user's Permissions tab and use this policy (change username if your user is not called jenkins):

{ "Version": "2012-10-17", "Statement": [ { "Sid": "Stmt1312295543082", "Action": [ "ec2:DescribeSpotInstanceRequests", "ec2:CancelSpotInstanceRequests", "ec2:GetConsoleOutput", "ec2:RequestSpotInstances", "ec2:RunInstances", "ec2:StartInstances", "ec2:StopInstances", "ec2:TerminateInstances", "ec2:CreateTags", "ec2:DeleteTags", "ec2:DescribeInstances", "ec2:DescribeInstanceTypes", "ec2:DescribeKeyPairs", "ec2:DescribeRegions", "ec2:DescribeImages", "ec2:DescribeAvailabilityZones", "ec2:DescribeSecurityGroups", "ec2:DescribeSubnets", "iam:ListInstanceProfilesForRole", "iam:PassRole", "ec2:GetPasswordData" ], "Effect": "Allow", "Resource": "*" } ] }

If you want to launch agents with an IAM Instance Profile, "iam:PassRole" permission is required.

If you want to launch Windows agents and use the generated Administrator password, the "ec2:GetPasswordData" permission is also required.

Configure plugin via Groovy script

Either automatically upon Jenkins post-initialization or through Jenkins script console, example:

import com.amazonaws.services.ec2.model.InstanceType import com.cloudbees.jenkins.plugins.awscredentials.AWSCredentialsImpl import com.cloudbees.plugins.credentials.* import com.cloudbees.plugins.credentials.domains.Domain import hudson.model.* import hudson.plugins.ec2.AmazonEC2Cloud import hudson.plugins.ec2.AMITypeData import hudson.plugins.ec2.EC2Tag import hudson.plugins.ec2.SlaveTemplate import hudson.plugins.ec2.SpotConfiguration import hudson.plugins.ec2.UnixData import jenkins.model.Jenkins import hudson.plugins.ec2.HostKeyVerificationStrategyEnum import hudson.plugins.ec2.ConnectionStrategy import hudson.plugins.ec2.Tenancy import hudson.plugins.ec2.EbsEncryptRootVolume def sshPortToConnectWith = '22' // store parameters def slaveTemplateUsEast1Parameters = [ ami: 'ami-AAAAAAAA', associatePublicIp: false, spotConfig: null, connectBySSHProcess: false, connectUsingPublicIp: false, customDeviceMapping: '', deleteRootOnTermination: true, description: 'Jenkins agent EC2 US East 1', ebsOptimized: true, iamInstanceProfile: '', idleTerminationMinutes: '5', initScript: '', instanceCapStr: '2', javaPath: 'java', jvmopts: '', labelString: 'aws.ec2.us.east.jenkins.worker', launchTimeoutStr: '', numExecutors: '1', unixData: new UnixData(null, null, null, sshPortToConnectWith, null), remoteFS: '', remoteAdmin: 'ec2-user', tmpDir: '', securityGroups: 'sg-11111111', stopOnTerminate: false, subnetId: 'subnet-SSSSSSSS', tags: new EC2Tag('Name', 'jenkins-worker'), type: 't2.medium', useDedicatedTenancy: false, useEphemeralDevices: false, usePrivateDnsName: false, userData: '', zone: '', metadataSupported: true, metadataEndpointEnabled: true, metadataTokensRequired: true, // `true` enforces IMDSv2 only (over IMDSv1), an important AWS security best practice metadataHopsLimit: 1, minimumNumberOfInstances: 0, minimumNumberOfSpareInstances: 0, maxTotalUses: -1, monitoring: false, t2Unlimited: false, connectionStrategy: ConnectionStrategy.valueOf('PRIVATE_IP'), hostKeyVerificationStrategy: HostKeyVerificationStrategyEnum.valueOf('CHECK_NEW_HARD'), tenancy: Tenancy.valueOf('Default'), ebsEncryptRootVolume: EbsEncryptRootVolume.valueOf('ENCRYPTED'), nodeProperties: null ] def AmazonEC2CloudParameters = [ name: 'MyCompany', credentialsId: 'jenkins-aws-key', instanceCapStr: '2', privateKey: '''-----BEGIN RSA PRIVATE

编辑推荐精选

酷表ChatExcel

酷表ChatExcel

大模型驱动的Excel数据处理工具

基于大模型交互的表格处理系统,允许用户通过对话方式完成数据整理和可视化分析。系统采用机器学习算法解析用户指令,自动执行排序、公式计算和数据透视等操作,支持多种文件格式导入导出。数据处理响应速度保持在0.8秒以内,支持超过100万行数据的即时分析。

AI工具酷表ChatExcelAI智能客服AI营销产品使用教程
DeepEP

DeepEP

DeepSeek开源的专家并行通信优化框架

DeepEP是一个专为大规模分布式计算设计的通信库,重点解决专家并行模式中的通信瓶颈问题。其核心架构采用分层拓扑感知技术,能够自动识别节点间物理连接关系,优化数据传输路径。通过实现动态路由选择与负载均衡机制,系统在千卡级计算集群中维持稳定的低延迟特性,同时兼容主流深度学习框架的通信接口。

DeepSeek

DeepSeek

全球领先开源大模型,高效智能助手

DeepSeek是一家幻方量化创办的专注于通用人工智能的中国科技公司,主攻大模型研发与应用。DeepSeek-R1是开源的推理模型,擅长处理复杂任务且可免费商用。

问小白

问小白

DeepSeek R1 满血模型上线

问小白是一个基于 DeepSeek R1 模型的智能对话平台,专为用户提供高效、贴心的对话体验。实时在线,支持深度思考和联网搜索。免费不限次数,帮用户写作、创作、分析和规划,各种任务随时完成!

AI主流办公工具有哪些办公热门AI 助手
KnowS

KnowS

AI医学搜索引擎 整合4000万+实时更新的全球医学文献

医学领域专用搜索引擎整合4000万+实时更新的全球医学文献,通过自主研发AI模型实现精准知识检索。系统每日更新指南、中英文文献及会议资料,搜索准确率较传统工具提升80%,同时将大模型幻觉率控制在8%以下。支持临床建议生成、文献深度解析、学术报告制作等全流程科研辅助,典型用户反馈显示每周可节省医疗工作者70%时间。

Windsurf Wave 3

Windsurf Wave 3

Windsurf Editor推出第三次重大更新Wave 3

新增模型上下文协议支持与智能编辑功能。本次更新包含五项核心改进:支持接入MCP协议扩展工具生态,Tab键智能跳转提升编码效率,Turbo模式实现自动化终端操作,图片拖拽功能优化多模态交互,以及面向付费用户的个性化图标定制。系统同步集成DeepSeek、Gemini等新模型,并通过信用点数机制实现差异化的资源调配。

AI IDE
腾讯元宝

腾讯元宝

腾讯自研的混元大模型AI助手

腾讯元宝是腾讯基于自研的混元大模型推出的一款多功能AI应用,旨在通过人工智能技术提升用户在写作、绘画、翻译、编程、搜索、阅读总结等多个领域的工作与生活效率。

AI助手AI对话AI工具腾讯元宝智能体热门 AI 办公助手
Grok3

Grok3

埃隆·马斯克旗下的人工智能公司 xAI 推出的第三代大规模语言模型

Grok3 是由埃隆·马斯克旗下的人工智能公司 xAI 推出的第三代大规模语言模型,常被马斯克称为“地球上最聪明的 AI”。它不仅是在前代产品 Grok 1 和 Grok 2 基础上的一次飞跃,还在多个关键技术上实现了创新突破。

OmniParser

OmniParser

帮助AI理解电脑屏幕 纯视觉GUI元素的自动化解析方案

开源工具通过计算机视觉技术实现图形界面元素的智能识别与结构化处理,支持自动化测试脚本生成和辅助功能开发。项目采用模块化设计,提供API接口与多种输出格式,适用于跨平台应用场景。核心算法优化了元素定位精度,在动态界面和复杂布局场景下保持稳定解析能力。

OmniParser界面解析交互区域检测Github开源项目
流畅阅读

流畅阅读

AI网页翻译插件 双语阅读工具,还原母语级体验

流畅阅读是一款浏览器翻译插件,通过上下文智能分析提升翻译准确性,支持中英双语对照显示。集成多翻译引擎接口,允许用户自定义翻译规则和快捷键配置,操作数据全部存储在本地设备保障隐私安全。兼容Chrome、Edge、Firefox等主流浏览器,基于GPL-3.0开源协议开发,提供持续的功能迭代和社区支持。

AI翻译AI翻译引擎AI翻译工具
下拉加载更多