Configuring Scaling Settings for Managed DevOps Pool : Fresh agent for every job and Standby Agents Mode is Off
Configuring Scaling Settings for Managed DevOps Pool : Fresh agent for every job and Standby Agents Mode is Off
To manage the performance and cost of your Managed DevOps Pool, I should configure the scaling settings appropriately. For more details on pricing and performance, refer to the official documentation:
Configure Scaling for Managed DevOps Pools
Scaling Configuration
In this tutorial, I have configured the Azure DevOps Managed DevOps Pool with the following scaling settings:
-
Pool Maximum Agents: 5
-
Agent State: Fresh agent for every job
-
Standby Agents Mode: Off (agents are provisioned on-demand when jobs are queued)
-
Parallel Jobs Capacity: 5 (enabled)
-
Running Builds: 5
With this setup, each build runs on a newly provisioned agent, ensuring a clean environment for execution. The use of 5 parallel self-hosted agents allows all builds to queue simultaneously , as long as the number of concurrent builds does not exceed the available capacity.
This configuration provides insights into key performance metrics such as queue delay, build duration, and machine reuse, which will be analyzed further to evaluate the overall efficiency of the setup.
A. WINDOWS : Azure Pipelines Windows Server 2022 image
Azure DevOps Build Analysis
1. Queue Delay (Queue Time → Start Time)
Queue delay is the time between when a build is queued and when an agent starts running it.
Build ID | Queue Time | Start Time | Queue Delay |
---|---|---|---|
1615 | 19:13:28 | 19:17:34 | 4 min 6 sec |
1616 | 19:13:31 | 19:17:40 | 4 min 9 sec |
1617 | 19:13:33 | 19:17:53 | 4 min 20 sec |
1618 | 19:13:35 | 19:17:59 | 4 min 24 sec |
1619 | 19:13:36 | 19:18:04 | 4 min 28 sec |
Queue delay is consistently around 4 to 4.5 minutes.
This suggests that:
- Fresh agent provisioning is the main bottleneck.
- No standby agents available leads to provisioning delay for every build.
2. Build Duration (Start Time → Finish Time)
Build duration is the time between when a build starts and when it completes.
Build ID | Start Time | Finish Time | Build Duration |
---|---|---|---|
1615 | 19:17:34 | 19:18:58 | 1 min 24 sec |
1616 | 19:17:40 | 19:19:04 | 1 min 24 sec |
1617 | 19:17:53 | 19:19:17 | 1 min 24 sec |
1618 | 19:17:59 | 19:19:38 | 1 min 39 sec |
1619 | 19:18:04 | 19:19:37 | 1 min 33 sec |
Most builds complete in ~1.5 minutes, which is much shorter than the queue delay.
This confirms that agent provisioning is a bigger issue than the build execution itself.
3. Machine Reuse
Since Agent State = Fresh Agent Every Time and Standby Agents Mode = Off, each build runs on a brand new VM.
- There is no machine reuse.
- Every build waits for a new VM to be provisioned, causing a ~4-minute delay.
4. Key Takeaways
Your builds are efficient once they start (~1.5 min per build).
Queue delay (~4.5 min) is much longer than build time.
Fresh agent mode without standby agents is the root cause of high delays.
B. UBUNTU : Azure Pipelines Ubuntu 22.04 image
Azure DevOps Build Analysis (Ubuntu)
1. Queue Delay (Queue Time → Start Time)
Queue delay is the time between when a build is queued and when an agent starts running it.
Build ID | Queue Time | Start Time | Queue Delay |
---|---|---|---|
1620 | 19:33:31 | 19:35:39 | 2 min 8 sec |
1621 | 19:33:33 | 19:35:49 | 2 min 16 sec |
1622 | 19:33:35 | 19:36:02 | 2 min 27 sec |
1623 | 19:33:37 | 19:36:36 | 2 min 59 sec |
1624 | 19:33:38 | 19:36:17 | 2 min 39 sec |
Queue delay is consistently around 2 to 3 minutes, which is shorter than the Windows queue delay (~4.5 min).
This suggests that Ubuntu agent provisioning is faster than Windows.
2. Build Duration (Start Time → Finish Time)
Build duration is the time between when a build starts and when it completes.
Build ID | Start Time | Finish Time | Build Duration |
---|---|---|---|
1620 | 19:35:39 | 19:36:30 | 51 sec |
1621 | 19:35:49 | 19:36:40 | 51 sec |
1622 | 19:36:02 | 19:36:55 | 53 sec |
1623 | 19:36:36 | 19:37:51 | 1 min 15 sec |
1624 | 19:36:17 | 19:37:10 | 53 sec |
Ubuntu builds complete faster (~50 sec) than Windows (~1.5 min).
Queue delay + build time is still dominated by agent provisioning.
3. Machine Reuse
Since Agent State = Fresh Agent Every Time and Standby Agents Mode = Off, each build runs on a brand new VM.
- There is no machine reuse.
- Every build waits for a new VM to be provisioned, causing a ~2.5-minute delay.
4. Key Takeaways
Ubuntu builds are faster than Windows (~50 sec vs. ~1.5 min).
Queue delay is lower for Ubuntu (~2.5 min) compared to Windows (~4.5 min).
Agent provisioning is still the main bottleneck.
C. COMPARISON : Azure Pipelines Windows Server 2022 image vs Azure Pipelines Ubuntu 22.04 image
Windows vs Ubuntu Build Performance in Azure DevOps
1. Queue Delay (Queue Time → Start Time)
Queue delay measures how long a build waits before execution.
OS | Build ID | Queue Time | Start Time | Queue Delay |
---|---|---|---|---|
Windows | 1615 | 19:13:28 | 19:17:34 | 4 min 6 sec |
Windows | 1616 | 19:13:31 | 19:17:40 | 4 min 9 sec |
Windows | 1617 | 19:13:33 | 19:17:53 | 4 min 20 sec |
Windows | 1618 | 19:13:35 | 19:17:59 | 4 min 24 sec |
Windows | 1619 | 19:13:36 | 19:18:04 | 4 min 28 sec |
Ubuntu | 1620 | 19:33:31 | 19:35:39 | 2 min 8 sec |
Ubuntu | 1621 | 19:33:33 | 19:35:49 | 2 min 16 sec |
Ubuntu | 1622 | 19:33:35 | 19:36:02 | 2 min 27 sec |
Ubuntu | 1623 | 19:33:37 | 19:36:36 | 2 min 59 sec |
Ubuntu | 1624 | 19:33:38 | 19:36:17 | 2 min 39 sec |
Ubuntu has a much shorter queue delay (~2.5 min) than Windows (~4.3 min).
Windows provisioning is slower, possibly due to OS startup overhead.
2. Build Duration (Start Time → Finish Time)
Build duration is the actual time a build takes to complete.
OS | Build ID | Start Time | Finish Time | Build Duration |
---|---|---|---|---|
Windows | 1615 | 19:17:34 | 19:18:58 | 1 min 24 sec |
Windows | 1616 | 19:17:40 | 19:19:04 | 1 min 24 sec |
Windows | 1617 | 19:17:53 | 19:19:17 | 1 min 24 sec |
Windows | 1618 | 19:17:59 | 19:19:38 | 1 min 39 sec |
Windows | 1619 | 19:18:04 | 19:19:37 | 1 min 33 sec |
Ubuntu | 1620 | 19:35:39 | 19:36:30 | 51 sec |
Ubuntu | 1621 | 19:35:49 | 19:36:40 | 51 sec |
Ubuntu | 1622 | 19:36:02 | 19:36:55 | 53 sec |
Ubuntu | 1623 | 19:36:36 | 19:37:51 | 1 min 15 sec |
Ubuntu | 1624 | 19:36:17 | 19:37:10 | 53 sec |
Ubuntu builds complete faster (~50 sec) compared to Windows (~1.5 min).
Ubuntu’s lower queue delay + shorter build time make it more efficient overall.
3. Key Differences
Metric | Windows Avg. | Ubuntu Avg. | Faster OS |
---|---|---|---|
Queue Delay | 4 min 17 sec | 2 min 30 sec | Ubuntu |
Build Duration | 1 min 29 sec | 54 sec | Ubuntu |
Total Time (Queue Delay + Build) | 5 min 46 sec | 3 min 24 sec | Ubuntu |
Ubuntu consistently outperforms Windows in queue delay and build duration.
Windows builds take ~2 min longer per build.
source code : Configuring Scaling Settings for Managed DevOps Pool