Configuring Scaling Settings for Managed DevOps Pool : Fresh agent for every job and Standby Agents Mode is Off

Configuring Scaling Settings for Managed DevOps Pool : Fresh agent for every job and Standby Agents Mode is Off

To manage the performance and cost of your Managed DevOps Pool, I should configure the scaling settings appropriately. For more details on pricing and performance, refer to the official documentation:

Configure Scaling for Managed DevOps Pools

Scaling Configuration

In this tutorial, I have configured the Azure DevOps Managed DevOps Pool with the following scaling settings:

  • Pool Maximum Agents: 5

  • Agent State: Fresh agent for every job

  • Standby Agents Mode: Off (agents are provisioned on-demand when jobs are queued)

  • Parallel Jobs Capacity: 5 (enabled)

  • Running Builds: 5

    scaling

With this setup, each build runs on a newly provisioned agent, ensuring a clean environment for execution. The use of 5 parallel self-hosted agents allows all builds to queue simultaneously , as long as the number of concurrent builds does not exceed the available capacity.

This configuration provides insights into key performance metrics such as queue delay, build duration, and machine reuse, which will be analyzed further to evaluate the overall efficiency of the setup.

A. WINDOWS : Azure Pipelines Windows Server 2022 image

Azure DevOps Build Analysis

1. Queue Delay (Queue Time → Start Time)

Queue delay is the time between when a build is queued and when an agent starts running it.

Build ID Queue Time Start Time Queue Delay
1615 19:13:28 19:17:34 4 min 6 sec
1616 19:13:31 19:17:40 4 min 9 sec
1617 19:13:33 19:17:53 4 min 20 sec
1618 19:13:35 19:17:59 4 min 24 sec
1619 19:13:36 19:18:04 4 min 28 sec

Queue delay is consistently around 4 to 4.5 minutes.
This suggests that:

  • Fresh agent provisioning is the main bottleneck.
  • No standby agents available leads to provisioning delay for every build.

2. Build Duration (Start Time → Finish Time)

Build duration is the time between when a build starts and when it completes.

Build ID Start Time Finish Time Build Duration
1615 19:17:34 19:18:58 1 min 24 sec
1616 19:17:40 19:19:04 1 min 24 sec
1617 19:17:53 19:19:17 1 min 24 sec
1618 19:17:59 19:19:38 1 min 39 sec
1619 19:18:04 19:19:37 1 min 33 sec

Most builds complete in ~1.5 minutes, which is much shorter than the queue delay.
This confirms that agent provisioning is a bigger issue than the build execution itself.


3. Machine Reuse

Since Agent State = Fresh Agent Every Time and Standby Agents Mode = Off, each build runs on a brand new VM.

  • There is no machine reuse.
  • Every build waits for a new VM to be provisioned, causing a ~4-minute delay.

4. Key Takeaways

Your builds are efficient once they start (~1.5 min per build).
Queue delay (~4.5 min) is much longer than build time.
Fresh agent mode without standby agents is the root cause of high delays.


B. UBUNTU : Azure Pipelines Ubuntu 22.04 image

Azure DevOps Build Analysis (Ubuntu)

1. Queue Delay (Queue Time → Start Time)

Queue delay is the time between when a build is queued and when an agent starts running it.

Build ID Queue Time Start Time Queue Delay
1620 19:33:31 19:35:39 2 min 8 sec
1621 19:33:33 19:35:49 2 min 16 sec
1622 19:33:35 19:36:02 2 min 27 sec
1623 19:33:37 19:36:36 2 min 59 sec
1624 19:33:38 19:36:17 2 min 39 sec

Queue delay is consistently around 2 to 3 minutes, which is shorter than the Windows queue delay (~4.5 min).
This suggests that Ubuntu agent provisioning is faster than Windows.


2. Build Duration (Start Time → Finish Time)

Build duration is the time between when a build starts and when it completes.

Build ID Start Time Finish Time Build Duration
1620 19:35:39 19:36:30 51 sec
1621 19:35:49 19:36:40 51 sec
1622 19:36:02 19:36:55 53 sec
1623 19:36:36 19:37:51 1 min 15 sec
1624 19:36:17 19:37:10 53 sec

Ubuntu builds complete faster (~50 sec) than Windows (~1.5 min).
Queue delay + build time is still dominated by agent provisioning.


3. Machine Reuse

Since Agent State = Fresh Agent Every Time and Standby Agents Mode = Off, each build runs on a brand new VM.

  • There is no machine reuse.
  • Every build waits for a new VM to be provisioned, causing a ~2.5-minute delay.

4. Key Takeaways

Ubuntu builds are faster than Windows (~50 sec vs. ~1.5 min).
Queue delay is lower for Ubuntu (~2.5 min) compared to Windows (~4.5 min).
Agent provisioning is still the main bottleneck.


C. COMPARISON : Azure Pipelines Windows Server 2022 image vs Azure Pipelines Ubuntu 22.04 image

Windows vs Ubuntu Build Performance in Azure DevOps

1. Queue Delay (Queue Time → Start Time)

Queue delay measures how long a build waits before execution.

OS Build ID Queue Time Start Time Queue Delay
Windows 1615 19:13:28 19:17:34 4 min 6 sec
Windows 1616 19:13:31 19:17:40 4 min 9 sec
Windows 1617 19:13:33 19:17:53 4 min 20 sec
Windows 1618 19:13:35 19:17:59 4 min 24 sec
Windows 1619 19:13:36 19:18:04 4 min 28 sec
Ubuntu 1620 19:33:31 19:35:39 2 min 8 sec
Ubuntu 1621 19:33:33 19:35:49 2 min 16 sec
Ubuntu 1622 19:33:35 19:36:02 2 min 27 sec
Ubuntu 1623 19:33:37 19:36:36 2 min 59 sec
Ubuntu 1624 19:33:38 19:36:17 2 min 39 sec

Ubuntu has a much shorter queue delay (~2.5 min) than Windows (~4.3 min).
Windows provisioning is slower, possibly due to OS startup overhead.


2. Build Duration (Start Time → Finish Time)

Build duration is the actual time a build takes to complete.

OS Build ID Start Time Finish Time Build Duration
Windows 1615 19:17:34 19:18:58 1 min 24 sec
Windows 1616 19:17:40 19:19:04 1 min 24 sec
Windows 1617 19:17:53 19:19:17 1 min 24 sec
Windows 1618 19:17:59 19:19:38 1 min 39 sec
Windows 1619 19:18:04 19:19:37 1 min 33 sec
Ubuntu 1620 19:35:39 19:36:30 51 sec
Ubuntu 1621 19:35:49 19:36:40 51 sec
Ubuntu 1622 19:36:02 19:36:55 53 sec
Ubuntu 1623 19:36:36 19:37:51 1 min 15 sec
Ubuntu 1624 19:36:17 19:37:10 53 sec

Ubuntu builds complete faster (~50 sec) compared to Windows (~1.5 min).
Ubuntu’s lower queue delay + shorter build time make it more efficient overall.


3. Key Differences

Metric Windows Avg. Ubuntu Avg. Faster OS
Queue Delay 4 min 17 sec 2 min 30 sec Ubuntu
Build Duration 1 min 29 sec 54 sec Ubuntu
Total Time (Queue Delay + Build) 5 min 46 sec 3 min 24 sec Ubuntu

Ubuntu consistently outperforms Windows in queue delay and build duration.
Windows builds take ~2 min longer per build.

source code : Configuring Scaling Settings for Managed DevOps Pool

Gora LEYE

I'm a microsoft most valuable professional (MVP) .NET Architect and Technical Expert skills located in Paris (FRANCE). The purpose of this blog is mainly to post general .NET tips and tricks, www.masterconduite.com Gora LEYE

Support us

BMC logoBuy me a coffee