Amazon Elastic Container Service (ECS / Fargate) Plugin for Jenkins
About
This Jenkins plugin uses Amazon Elastic Container Service to host jobs execution inside docker containers.
Jenkins delegates to Amazon ECS the execution of the builds on Docker based agents. Each Jenkins build is executed on a dedicated Docker container that is wiped-out at the end of the build.
- use GitHub Issues to report issues / feature requests
Installation & configuration
The scope of the plugin is only using existing and pre-configured AWS Infrastructure. It does not create any of the needed infrastructure on its own. Use tools like CloudFormation or Terraform for this task.
Requirements
- Jenkins with at least version 2.289.1
- AWS Account
Plugin install
Use the Jenkins plugin manager to install the Amazon Elastic Container Service plugin
Configuration
Examples
There are currently the following example setups in this repo:
- Fargate - ephemeral master and agents on Fargate
- Fargate with CDK (by AWS) - Jenkins Setup designed by AWS
Additionally there is an example setup here: Terraform Jenkins AWS ECS Fargate
Amazon ECS cluster
As a pre-requisite, you must have created an Amazon ECS cluster with associated ECS instances. These instances can be statically associated with the ECS cluster or can be dynamically created with Amazon Auto Scaling.
The Jenkins Amazon EC2 Container Service plugin will use this ECS cluster and will create automatically the required Task Definition.
Jenkins System Configuration
Navigate to the "Configure System" screen.
In the "Jenkins Location" section, ensure that the "Jenkins URL" is reachable from the the container instances of the Amazon ECS cluster. See the section "Network and firewalls" for more details.
If the global Jenkins URL configuration does not fit your needs (e.g. if your ECS agents must reach Jenkins through some kind of tunnel) you can also override the Jenkins URL in the Advanced Configuration of the ECS cloud.
At the bottom of the screen, click on "Add a new Cloud" and select "Amazon EC2 Container Service Cloud".
Amazon EC2 Container Service Cloud
Then enter the configuration details of the Amazon EC2 Container Service Cloud:
Name
: name for your ECS cloud (e.g.ecs-cloud
)Amazon ECS Credentials
: Amazon IAM Access Key with privileges to create Task Definitions and Tasks on the desired ECS clusterECS Cluster
: desired ECS cluster on which Jenkins will send builds as ECS tasksECS Template
: click on "Add" to create the desired ECS template or templates
Advanced configuration
Tunnel connection through
: tunnelling options (when Jenkins runs behind a load balancer).Alternative Jenkins URL
: The URL used as the Jenkins URL within the ECS containers of the configured cloud. Can be used to override the default Jenkins URL from global configuration if needed.
ECS Agent Templates
One or several ECS agent templates can be defined for the Amazon EC2 Container Service Cloud. The main reason to create more than one ECS agent template is to use several Docker images to perform build (e.g. java-build-tools, php-build-tools...)
Template name
is used (prefixed with the cloud's name) for the task definition in ECS.Label
: agent labels used in conjunction with the job level configuration "Restrict where the project can be run / Label expression". ECS agent label could identify the Docker image used for the agent (e.g.docker
for the jenkinsci/inbound-agent). Multiple, space delimited labels can be specified(e.g.java11 alpine
). Label expressions within a job such asjava11 && alpine
orjava11 || alpine
are not currently supported.Filesystem root
: working directory used by Jenkins (e.g./home/jenkins/
).Memory
: number of MiB of memory reserved for the container. If your container attempts to exceed the memory allocated here, the container is killed.CPU units
: number ofcpu units
to reserve for the container. A container instance has 1,024 cpu units for every CPU core.
Advanced Configuration
Override entrypoint
: overwritten Docker image entrypoint. Container command can't be overriden as it is used to pass jenkins agent connection parameters.JVM arguments
: additional arguments for the JVM, such as-XX:MaxPermSize
or GC options.
Network and firewalls
Running the Jenkins master and the ECS container instances in the same Amazon VPC and in the same subnet is the simplest setup and default settings will work out-of-the-box.
Firewalls If you enable network restrictions between the Jenkins master and the ECS cluster container instances,
- Fix the TCP listen port for JNLP agents of the Jenkins master (e.g.
5000
) navigating in the "Manage Jenkins / Configure Global Security" screen - Allow TCP traffic from the ECS cluster container instances to the Jenkins master on the listen port for JNLP agents (see above) and the HTTP(S) port.
Network Address Translation and Reverse Proxies In case of Network Address Translation rules between the ECS cluster container instances and the Jenkins master, ensure that the JNLP agents will use the proper hostname to connect to the Jenkins master doing on of the following:
- Define the proper hostname of the Jenkins master defining the system property
hudson.TcpSlaveAgentListener.hostName
in the launch command - Use the advanced configuration option "Tunnel connection through" in the configuration of the Jenkins Amazon EC2 Container Service Cloud (see above).
IAM Permissions
To work the plugin needs some IAM permissions. Assign a role with those permissions to the instance / container you are running the master on.
Here is an example of a role in CloudFormation, make sure to modify it for your needs.
TaskRole: Type: AWS::IAM::Role Properties: RoleName: !Sub ${AWS::StackName}-task-role Path: / AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - ecs-tasks.amazonaws.com Action: sts:AssumeRole Policies: - PolicyName: !Sub ecs-${AWS::StackName} PolicyDocument: Version: "2012-10-17" Statement: - Action: - "ecs:RegisterTaskDefinition" - "ecs:ListClusters" - "ecs:DescribeContainerInstances" - "ecs:ListTaskDefinitions" - "ecs:DescribeTaskDefinition" - "ecs:DeregisterTaskDefinition" - "ecs:ListTagsForResource" Effect: Allow Resource: "*" - Action: - "ecs:ListContainerInstances" - "ecs:DescribeClusters" Effect: Allow Resource: - !Sub "arn:aws:ecs:${AWS::Region}:${AWS::AccountId}:cluster/<clusterName>" - Action: - "ecs:RunTask" Effect: Allow Condition: ArnEquals: ecs:cluster: - !Sub "arn:aws:ecs:${AWS::Region}:${AWS::AccountId}:cluster/<clusterName>" Resource: !Sub "arn:aws:ecs:${AWS::Region}:${AWS::AccountId}:task-definition/*" - Action: - "ecs:StopTask" Effect: Allow Condition: ArnEquals: ecs:cluster: - !Sub "arn:aws:ecs:${AWS::Region}:${AWS::AccountId}:cluster/<clusterName>" Resource: !Sub "arn:aws:ecs:*:*:task/*" # "arn:aws:ecs:${AWS::Region}:${AWS::AccountId}:task/*" - Action: - "ecs:DescribeTasks" Effect: Allow Condition: ArnEquals: ecs:cluster: - !Sub "arn:aws:ecs:${AWS::Region}:${AWS::AccountId}:cluster/<clusterName>" Resource: !Sub "arn:aws:ecs:*:*:task/*" # "arn:aws:ecs:${AWS::Region}:${AWS::AccountId}:task/*" - Action: - "elasticfilesystem:DescribeAccessPoints" - "elasticfilesystem:DescribeFileSystems" Effect: Allow Resource: !Sub "arn:aws:elasticfilesystem:${AWS::Region}:${AWS::AccountId}:file-system/*"
Agent
The Jenkins Amazon EC2 Container Service Cloud can use for the agents all the Docker image designed to act as a Jenkins JNLP agent. Here is a list of compatible Docker images:
You can easily extend the images or also build your own.
Declarative Pipeline
Declarative Pipeline support requires Jenkins 2.66+
Declarative agents can be defined like shown below. You can also reuse pre-configured templates and override certain settings using inheritFrom
to reference the Label field
of the template that you want to use as preconfigured. Only one label is expected to be specified.
When using inheritFrom, the label will not copied. Instead, a new label will be generated based on the following schema {job-name}-{job-run-number}-{5-random-chars} e.g. "pylint-543-b4f42". This guarantees that there will not be conflicts with the parent template or other runs of the same job, as well as making it easier to identify the labels in Jenkins.
If you want to override the label, ensure that you are not going to conflict with other labels configured elsewhere. Templates for dynamic agents exist until the agent dies, meaning other jobs requesting the same label (including dynamic agents on other runs of the same job!) run the chance of provisioning the dynamic agent's ECSTask.
Note: You have to configure list of settings to be allowed in the declarative pipeline first (see the Allowed Overrides setting). They are disabled by default for security reasons, to avoid non-privileged users to suddenly be able to change certain settings.
If Jenkins is unexpectedly shut down there is a good chance that ECS Tasks for dynamic agents will not be cleaned up (de-registered) in AWS. This should not cause issues, but may come as a surprise when looking at the console.
Usage
The ECS agents can be used for any job and any type of job (Freestyle job, Maven job, Workflow job...), you just have to restrict the execution of the jobs on one of the labels used in the ECS Agent Template configuration. You can either restrict the job to run on a specific label only via the UI or directly in the pipeline.
pipeline { agent none stages { stage('PublishAndTests') { environment { STAGE='prod' } agent { label 'build-python36' } } steps { sh 'java -version' } } }
pipeline { agent none stages { stage('Test') { agent { ecs { inheritFrom 'label-of-my-preconfigured-template' cpu 2048 memory 4096 image '$AWS_ACCOUNT.dkr.ecr.$AWS_REGION.amazonaws.com/jenkins/java8:2019.7.29-1' logDriver 'fluentd' logDriverOptions([[name: 'foo', value:'bar'], [name: 'bar', value: 'foo']]) portMappings([[containerPort: 22, hostPort: 22, protocol: 'tcp'], [containerPort: 443, hostPort: 443, protocol: 'tcp']]) } } steps { sh 'echo hello' } } } }
Scripted Pipeline examples
def dynamic_label = "${JOB_NAME}_${env.sha}" ecsTaskTemplate( cloud: 'CloudNameAsConfiguredInManageClouds', label: dynamic_label, name: dynamic_label, // Reusing the label as a name makes sense as long as it's unique containerUser: 'ubuntu', remoteFSRoot: '/home/ubuntu', overrides: [], agentContainerName: 'java', taskDefinitionOverride: "arn:aws:redacted:redacted:task-definition/${env.task}" ) { node(dynamic_label) { stage("I dunno why you say goodbye"){ sh 'echo hello' } } }
pipeline{ agent { ecs { inheritFrom 'ecs_test' cpu 1000 } } stages{ stage("Here goes nothin"){ sh 'echo hello' } } }
FAQ
My parallel jobs don't start at the same time
Actually, there can be multiple reasons:
-
The plugin creates a new agent only when the stage contains an
agent
definition. If this is missing, the stage inherits the agent definition from the level above and also re-uses the instance. -
Also, parallel stages sometimes don't really start at the same time. Especially, when the provided label of the
agent
definition is the same. The reason is that Jenkins tries to guess how many instances are really needed and tells the plugin to start n instances of the agent with label x. This number is likely smaller than the number of parallel stages that you've declared in your Jenkinsfile. Jenkins calls the ECS plugin multiple times to get the total number of agents running. -
If launching of the agents takes long, and Jenkins calls the plugin in the meantime again to start n instances, the ECS plugin doesn't know if this instances are really needed or just requested because of the slow start. That's why the ECS plugin subtracts the number of launching agents from the number of requested agents (for a specific label). This can mean for parallel stages that some of the agents are launched after the previous bunch of agents becomes online.
There are options that influence how Jenkins spawns new Agents. You can set for example on your master the following to improve the launch times:
-Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
Who runs this & Resources
If you are running a interesting setup or have public posts abour your setups using this plugins, please file a PR to get it added here.
- Slides: Run Jenkins as managed product on ECS
- Youtube: Jenkins with Amazon ECS slaves
- AWS Blog - Jenkins on AWS
Maintainers
Andreas Sieferlinger (GitHub Twitter)
Philipp Garbe (GitHub, Twitter)
Marky Jackson (GitHub, Twitter)
Stephen Erickson (GitHub)
Developing
Building the Plugin
java -version # Need Java 1.8, earlier versions are unsupported for build mvn -version # Need a modern maven version; maven 3.2.5 and 3.5.0 are known to work mvn clean install
Running locally
To run locally, execute the following command and open the browser http://localhost:8080/jenkins/
mvn -e hpi:run
Debugging the plugin in an editor
IntelliJ IDEA
In the Maven dialog right click hpi:run
and select Debug
.
The IDE will stop at any breakpoints you have set inside the plugin.
Other
the
@Rule public JenkinsRule j = new JenkinsRule();
Will actually invoke code that will bootstrap a local installation of jenkins.war
. This will allow you to debug with with breakpoints and such. However, to do it
you will need to set some system properties or be aware how it tries to auto-configure. It will attempt to look for a .jenkins
directory recursively with an already exploded war,
So, theoretically you explode it, and git ignore it, right in this space. Alternatively, you can set a System property:
-Djth.jenkins-war.path=${PATH}/jenkins.war
Make sure to include this rule in any tests that touch Jenkins specific resources like: Jenkins.instance()
Releasing the Plugin
mvn clean release:prepare release:perform
further checks to aid with development
Check for additional or forgotten dependencies:
mvn dependency:analyze
Check if javadoc works fine (usually only executed on release)
mvn
编辑推荐精选

酷表ChatExcel
大模型驱动的Excel数据处理工具
基于大模型交互的表格处理系统,允许用户通过对话方式完成数据整理和可视化分析。系统采用机器学习算法解析用户指令,自动执行排序、公式计算和数据透视等操作,支持多种文件格式导入导出。数据处理响应速度保持在0.8秒以内,支持超过100万行数据的即时分析。


DeepEP
DeepSeek开源的专家并行通信优化框架
DeepEP是一 个专为大规模分布式计算设计的通信库,重点解决专家并行模式中的通信瓶颈问题。其核心架构采用分层拓扑感知技术,能够自动识别节点间物理连接关系,优化数据传输路径。通过实现动态路由选择与负载均衡机制,系统在千卡级计算集群中维持稳定的低延迟特性,同时兼容主流深度学习框架的通信接口。


DeepSeek
全球领先开源大模型,高效智能助手
DeepSeek是一家幻方量化创办的专注于通用人工智能的中国科技公司,主攻大模型研发与应用。DeepSeek-R1是开源的推理模型,擅长处理复杂任务且可免费商用。


问小白
DeepSeek R1 满血模型上线
问小白是一个基于 DeepSeek R1 模型的智能对话平台,专为用户提供高效、贴心的对话体验。实时在线,支持深度思考和联网搜索。免费不限次数,帮用户写作、创作、分析和规划,各种任务随时完成!


KnowS
AI医学搜索引擎 整合4000万+实时更新的全球医学文献
医学领域专用搜索引擎整合4000万+实时更新的全球医学文献,通过自主研发AI模型实现精准知识检索。系统每日更新指南、中英文文献及会议资料,搜索准确率较传统工具提升80%,同时将大模型幻觉率控制在8%以下。支持临床建议生成、文献深度解析、学术报告制作等全流程科研辅助,典型用户反馈显示每周可节省医疗工作者70%时间。


Windsurf Wave 3
Windsurf Editor推出第三次重大更新Wave 3
新增模型上下文协议支持与智能编辑功能。本次更新包含五项核心改进:支持接入MCP协议扩展工具生态,Tab键智能跳转提升编码效率,Turbo模式实现自动化终端操作,图片拖拽功能优化多模态交互,以及面向付费用户的个性化图标定制。系统同步集成DeepSeek、Gemini等新模型,并通过信用点数机制实现差异化的资源调配。


腾讯元宝
腾讯自研的混元大模型AI 助手
腾讯元宝是腾讯基于自研的混元大模型推出的一款多功能AI应用,旨在通过人工智能技术提升用户在写作、绘画、翻译、编程、搜索、阅读总结等多个领域的工作与生活效率。


Grok3
埃隆·马斯克旗下的人工智能 公司 xAI 推出的第三代大规模语言模型
Grok3 是由埃隆·马斯克旗下的人工智能公司 xAI 推出的第三代大规模语言模型,常被马斯克称为“地球上最聪明的 AI”。它不仅是在前代产品 Grok 1 和 Grok 2 基础上的一次飞跃,还在多个关键技术上实现了创新突破。


OmniParser
帮助AI理解电脑屏幕 纯视觉GUI元素的自动化解析方案
开源工具通过计算机视觉技术实现图形界面元素的智能识别与结构化处理,支持自动化测试脚本生成和辅助功能开发。项目采用模块化设计,提供API接口与多种输出格式,适用于跨平台应用场景。核心算法优化了元素定位精度,在动态界面和复杂布局场景下保持稳定解析能力。


流畅阅读
AI网页翻译插件 双语阅读工具,还原母语级体验
流畅阅读是一款浏览器翻译插件,通过上下文智能分析提升翻译准确性,支持中英双语对照显示。集成多翻译引擎接口,允许用户自定义翻译规则和快捷键配置,操作数据全部存储在本地设备保障隐私安全。兼容Chrome、Edge、Firefox等主流浏览器,基于GPL-3.0开源协议开发,提供持续的功能迭代和社区支持。
推荐工具精选
AI云服务特惠
懂AI专属折扣关注微信公众号
最新AI工具、AI资讯
独家AI资源、AI项目落地
