The early returns on gen AI for software development

As development teams become increasingly familiar with these tools, their understanding of the tools and ability to apply them to diverse business scenarios will bring even greater value, Lanehart says.

“For example, being able to completely communicate an entire application request to gen AI that generates all necessary code will reduce a task timeline significantly,” Lanehart says. That means an engineer or team is freed up to spend more time thinking creatively or strategically about the overall project and how to further improve it, he says.

The development team at software company ZoomInfo was eager to experiment with gen AI tools once they became available last year for use at scale, says CTO Ali Dasdan.

One of the earliest use cases provided the company’s software developers with access to Github Copilot. After initial success, ZoomInfo has begun to integrate gen AI throughout its organization to improve productivity, Dasdan says. One example is with document search and summarization.

“Software development requires heavy documentation,” Dasdan says. “Documents such as product requirements and architecture designs are standard in well-run organizations,” but development teams need a lot of time to review these documents. “Gen AI has freed up a significant amount of time by summarizing and indexing these documents in just a few minutes,” he says.

The success of the trial led the company to get licenses for nearly all its software developers. “Our engineers still have to review the code the tool creates, however,” Dasdan says. “We’ve already accepted tens of thousands of lines of code and we’ve realized a significant amount of time saved.”

Limits, team impacts, and lessons learned

One of the key takeaways from early use of gen AI is that it won’t replace human developers.

At IT services provider BDO Digital, initially there was a “wave of excitement” about gen AI’s potential to autonomously generate complex software, says Kirstie Tiernan, principal in the firm’s data and AI practice.

“However, we quickly learned that AI is a tool to augment human expertise, not replace it,” Tiernan says. “The need for human oversight to ensure the quality and functionality of AI-generated code quickly became apparent. It’s a partnership where AI handles some of the heavy lifting, allowing developers to focus on strategic problem-solving.”

A key lesson BDO is working through is the importance of integrating AI tools with existing workflows. “It isn’t just about adopting new tools; [it’s] more about how development teams operate, communicate, and collaborate,” Tiernan says. “The integration process highlights the need for flexibility and adaptability in all of our development practices.”

One of the more interesting surprises at BDO was the impact of gen AI on creativity and innovation. “By automating routine tasks, developers have been freed up to tackle more complex challenges and explore more innovative solutions,” Tiernan says. “It’s exciting to see how AI can serve as a catalyst for human creativity and ideation.”

With any new technology solution, one of the biggest challenges is identifying the extent to which a team should integrate or rely on the tool, Momnt’s Lanehart says.

“One of our core beliefs is using technology to empower and support people,” he says. “So, we knew that we did not want AI and gen AI tools to function as a replacement for our employees. Instead, we wanted the tools to complement the skills that these individuals bring to the team and help them function more effectively and efficiently.”

The technology industry overall is seeing more demand for people who can oversee, implement, and run gen AI tools, Lanehart says. For example, this could be an employee who functions on the product development team, but whose core expertise is ChatGPT or Copilot, he says.

As gen AI becomes increasingly prevalent, “we’re seeing value in having people who are cross-trained with these tools,” Lanehart says. “We want people who can solve new problems in diverse ways, and we want them to bring that knowledge back to our team.”

Momnt has begun encouraging its software development team members to expand their understanding of gen AI tools by applying them to their personal interests, such as music, comedy, and other areas, Lanehart says. “Finding overlaps between the applications of AI to both fintech and personal interest puts our team in a unique position to drive new industry growth,” he says.

Lyric, a healthcare technology company, is harnessing the power of LLMs to improve several processes, says Akshay Sharma, chief AI officer. But one of the early lessons was how much work was needed to get the correct value from LLMs.

“Out of the box they are somewhat generalized, miss the mark, and hallucinate,” Sharma says. “But, with the right engineering and design [and by] running experiments with prompts, we get a lot of mileage out of this. We had to build a lot of experimental and testing frameworks to continuously evaluate gen AI.”

Freshworks’ Ramakrishnan believes Gen AI has the potential to enable developers to bring applications to market faster; “however, their skillset will need to adjust to be professionals at prompt engineering,” he says. “AI-generated coding will only be as valuable and accurate as the type of prompt that is asked.”

Moreover, AI code needs to be verified by experienced developers to confirm accuracy, Ramakrishnan adds. “I can’t overemphasize the value of code reviews by humans on machine-generated codes,” he says. “Despite its productivity value to the workplace, AI is far from perfect and requires oversight.”

In addition, the use of AI raises some ethical issues related to the introduction of bias in algorithms, which can lead to unintended consequences if not checked, Ramakrishnan says. “It also introduces new considerations in the area of information security,” he says. “Bad actors now have a broader reach for introducing malicious code in millions if not billions of systems.”

There will be a constant need to re-tool the workforce to make effective use of AI, Ramakrishnan. “This said, we are barely scratching the surface of gen AI’s productivity value,” he says. “The best days are ahead of us.”

Taking a developers’-eye view

One of the best ways to determine the impact of gen AI on development teams is to ask team members to weigh in on their experiences. To collect internal feedback on ZoomInfo’s use of GitHub Copilot, the company conducted a survey of about 80 of its developers. The research showed that Copilot has several strengths.

One is the ability to generate boilerplate and repetitive code, which enables developers to focus on complex logic. Another is a drastic reduction in the time it takes to write unit tests. “Many users report that the tool improves their coding speed by offering useful code suggestions and auto-completing lines,” ZoomInfo’s Dasdan says.

These strengths resulted in several benefits for ZoomInfo’s developers, with a large majority saying Copilot reduced the amount of time it takes to complete tasks, by an average of 20%. About two thirds said using the gen AI technology allowed them to complete more tasks per sprint, and about three quarters said the quality of their work was improved.

“Based on these strong early results, we anticipate gen AI tools continuing to improve the productivity of our engineers, as well as save time from tasks that are auxiliary to writing actual production code,” Dasdan says.

Software development’s gen AI future

Development leaders are confident that gen AI will only grow in importance as a development tool.

“Looking ahead, the potential for productivity gains with generative AI is substantial,” BDO Digital’s Tiernan says. “As these tools become more integrated into the fabric of software development, we’re likely to see dramatic reductions in development time and costs.”

For example, automating the generation of boilerplate code and providing real-time suggestions for bug fixes can halve the time traditionally needed for certain development tasks, Tiernan says.

“But the real game changer will be in how generative AI enables us to tackle more complex problems more efficiently,” Tiernan says. “With AI handling routine aspects, developers can focus on strategic innovation, pushing the boundaries of what’s possible in software solutions.”