Gen AI success starts with an effective pilot strategy

Looking to achieve internal efficiency opportunities with gen AI, Skillsoft began with a Digital and IT Hackathon focused on leveraging ChatGPT to solve business problems or create new business value, Daly says. “But the real goal was to start to get people familiar with leveraging ChatGPT in a safe space, understand its potential and shortcomings in the context of real business use cases,” she says.

Since then, team members across product, engineering, content production, and the lines of business have been engaged in pilots and bringing new capabilities to market leveraging gen AI.

“In many examples, we have quickly moved from the pilot phase to new customer offerings or platform enhancements enabled by the responsible use of AI,” Daly says.

Skillsoft’s CAISY (Conversation AI Simulator), its gen AI-based tool designed to help professionals develop their business and leadership conversation skills, is an example of that.

The company will continue identifying opportunities to use gen AI to

deliver new capabilities. “We plan to leverage AI for driving adaptive learner journeys — meeting the learner where they are at — as well as to derive insights from usage data for [administrators] to understand how to best improve engagement and performance,” Daly says.

Skillsoft recently launched an Innovation Challenge, inviting all employees

to participate and get familiar with gen AI capabilities. “The winning solution from 18 teams will be showcased at the company town hall in September, highlighting the importance of embracing AI and making it part of our culture of innovation,” Daly says.

The company has also established an AI Advancement Council, with cross-functional representation. It’s charged with facilitating and advancing the integration of AI technology at Skillsoft, providing a collaborative forum and ensuring AI is effectively and responsibly used to drive innovation, efficiency, and growth, Daly says.

“We encourage all our team members to take advantage of our expansive content library, which offers a solid foundation on topics required for AI and AI-adjacent roles,” Daly says.

Democratizing AI education and ideation

Ceridian, a provider of human resources software, began its foray into gen AI by applying it within its Dayforce human capital management platform as a pilot program. Specifically, the company is using AI for self-service capabilities to enhance customer support.

The initiative was driven in the first quarter of 2023 by leaders within the customer support organization, says David Lloyd, chief data officer.

“Today, we are fine-tuning the model, using it internally first to achieve a strong correct answer rate,” Lloyd says. “Given our success in increasing the rate of answering directed questions to 85%, we are now working towards another 10% increase. In the future, we see the ability to use this approach for additional customer self-service as well as other resources.”

All of this is a byproduct of training the model against implementation guides, knowledge bases, and other internal documentation that is refined against customer questions and responses, Lloyd says.

Hundreds of internal customer support professionals are running all questions asked by customers and themselves through the platform, to determine the accuracy of the response from a reinforcement learning perspective, Lloyd says. “It will likely continue to be an internal support assistant as we consider when it can be provided to customers,” he says.

The key for Ceridian is democratizing how employees and partners think of AI. “We are focused on education, simple AI idea submission, and weekly ‘ask me anything’ sessions to discuss many topics,” Lloyd says.

The company’s Dayforce Lab provides an environment for testing and validating new AI technologies, as well as coaching and mentoring for business and product teams. “Dayforce Labs is complemented by our data science team, as there is no AI without data,” Lloyd says. “This includes carefully examining how data can be used to create better outcomes through AI.”

Every Ceridian employee can have a positive impact on the use of AI technologies, Lloyd says, whether it’s within internal business groups or via Dayforce. “In the end, we need to ensure that AI is being used to produce an outcome that has business value and supports our core principles,” he says.

Best practices for gen AI pilot success

For those enterprises looking to deploy gen AI starting with pilot programs, a few practices are worth considering.

Before introducing generative AI in the workplace, leaders should begin with a risk assessment to identify what risks are present to them as an organization in the use and potential misuse of the tool, Daly says. This could be misappropriation, plagiarism, or bias, to name a few.

“Establishing a sustainable, trustworthy, and transparent governance structure requires shared accountability from the company, its employees, and its providers,” Daly says.

Because the responsibility shouldn’t fall on one person or department, leaders might want to establish a cross-functional team to govern and help drive adoption, as well as create opportunities to identify use cases, Daly says. “Good use cases can come from anywhere in the organization, but need support to go from a good idea to a prototype or a production-ready solution,” she says.

In addition, Daly says, technology leaders should provide team members with the training and tools to get educated and stay current on the capabilities of gen AI. “Promote a culture of curiosity and learning, to keep pace with what is a fast-evolving space,” she says.

IT leaders should create a “safe space” for innovation, Iacob says. “It’s vital to encourage innovation without stifling creativity due to concerns around data protection risks or the use of proprietary data,” he says. “By providing clear training and guidelines, you can establish boundaries that protect essential assets while still allowing engineers and other team members the freedom to explore and experiment.”

This nurturing environment not only fosters innovation, but also builds confidence, ensuring that potential risks are well-managed without limiting the possibilities that gen AI offers, Iacob says.

It’s important to keep in mind that not all large language models are the same, Lloyd says. “Some have different attributes in the way they were trained,” he says. “This is important, as it may shape how you deploy the model. For example, do you want it to be more information than conversational? Do you want to use one that’s more fine-tuned for a specific industry or knowledge domain?”

Another good practice is to consider multiple models and stay up to date on technology developments.

“The gen AI space is evolving at a rapid pace, with new models and iterations frequently becoming available,” Iacob says. “It’s essential to maintain an open and explorative mindset, considering various models that fit your specific needs, and continuously staying abreast of the latest developments.”

By doing so, companies can position themselves to benefit from advancements in the field, tailoring solutions that are both innovative and aligned with the latest best practices, Iacob says.