Engineering leaders back AI gains, but measurement lags

Uplevel has published survey findings showing that most Engineering and Technology leaders believe AI is improving software development, but far fewer use controlled comparisons to test that belief.

The survey of 428 Engineering and Technology leaders found that 96% were confident AI was responsible for the results they were seeing, while 40% said they used controlled comparisons between teams using AI and those that were not. The research examined how engineering organisations measure AI's value, attribute performance improvements and assess barriers to wider use.

Measurement gap

The findings highlight a gap between confidence in AI and the methods used to assess its effect. Nearly three-quarters of respondents, 74%, said they had extended measurement approaches that pre-dated AI to cover AI initiatives, while 4% said they were building new approaches specifically for AI.

Only 29% said they primarily evaluated AI's value through business outcomes, suggesting that many teams still focus on operational indicators rather than broader commercial impact.

"Engineering leaders are seeing real benefits from AI, but confidence alone doesn't tell us how much of that improvement is driven by AI rather than other influences," said Joe Levy, Chief Executive Officer of Uplevel.

"As organizations invest more heavily in AI, they need a clearer picture of its impact: one that shows where the gains are coming from, what's potentially holding them back, and whether they're translating into meaningful business outcomes," Levy said.

Speed and friction

Speed remained the main measure many leaders would use to decide whether AI creates value. The survey found that 39% identified speed as the primary factor they would consider, and the same share said AI mainly creates value by helping developers write code faster.

That pattern held across seniority levels, with 41% of Managers, 40% of Directors and 40% of Vice-Presidents naming speed as the main consideration.

However, the data also suggested that faster code generation can create extra work elsewhere in the software development process. Nearly three in 10 respondents (29%) said increased time spent reviewing AI-generated work had been their biggest new source of friction over the past quarter, a higher share than for any other challenge listed in the research.

Amy Carrillo Cotten, Director of Client Transformation at Uplevel, said the results showed that coding speed alone did not determine delivery outcomes.

"AI is making parts of software development faster, and we've always been told that time is money," said Amy Carrillo Cotten, Director of Client Transformation at Uplevel.

"But faster coding doesn't automatically mean faster or better delivery. As AI accelerates certain tasks, it's also exposing where work gets stuck. Engineering leaders need to look at the entire delivery process - not just how quickly code gets written - to understand where improvements will make the biggest difference," Cotten said.

Maturity levels

The research also grouped organisations by AI maturity on a five-point scale, ranging from routine use of code-completion tools to software deployment with minimal human intervention. Responses showed a wide spread across those stages.

Some 38% were in the earlier stages, covering organisations that consistently use AI for tasks such as code completion or use AI agents in specific, limited areas. Another 23% were in the middle of the scale, where organisations configure AI agents with internal context such as codebases, conventions and runbooks.

A further 38% were at more advanced stages. In those organisations, teams develop, maintain and share AI agents across departments, while the most advanced respondents described workflows built around orchestrated agent teams that can deploy software independently with limited human input.

Changing barriers

Barriers to extracting more value from AI varied by stage of adoption. Across the full sample, governance, security and compliance concerns were the most commonly cited obstacle, at 24%.

Gaps in engineering skills or AI literacy and missing context, such as documentation, data or institutional knowledge, each accounted for 21%. Workflow friction and infrastructure limits both stood at 17%.

Earlier-stage organisations were more likely to point to missing context as the main problem. Among Level 1 organisations, 25% named it as the biggest impediment, rising to 27% at Level 2.

At more advanced stages, the emphasis shifted. Skills and AI literacy gaps were cited by 26% of Level 4 organisations and 30% of Level 5 organisations, while governance, security and compliance concerns were identified by 28% of Level 4 and 26% of Level 5 respondents.

The distribution suggests that as companies embed AI more deeply into development work, the issues shift from access to useful context to questions of oversight, control and workforce readiness.

The survey also indicated that no single obstacle dominated across all organisations, pointing instead to a fragmented landscape in which AI adoption is shaped by a mix of technical, operational and organisational constraints. For Engineering leaders, that leaves a central challenge: showing whether AI is improving software delivery in ways that can be measured rather than assumed.