Who Tests the Code AI Writes at Insurance Carriers?

Introduction

Carriers are rewriting decades-old systems, and more and more of the new code is written with AI. If the AI that wrote the code is also the thing checking it, who is actually validating what reaches the policyholder?

Alex Rodov has spent more than thirty years in software testing, and his answer is that the testing has to be independent and has to start from a complete map of how a carrier's systems actually work. He explains why most carrier testing is still manual, why Emtech keeps AI out of the testing itself, and why the same map turns out to be what a carrier's enterprise AI has been missing.

Guest Bio

Alex Rodov is the founder of Emtech Group, which builds autonomous software testing for insurance carriers and insurtechs, starting with life insurance. He founded QA Consultants in 1994 and built it into one of the largest independent software testing firms in North America. When ALTEN acquired the firm in 2023, he carved out its emerging technology division, a team of PhDs and engineers building testing accelerators, and it became Emtech. He is the author of The Book on Testing and sits on LIMRA's Strategy and Innovation Committee.

Key Topics

-Who checks the AI's work - If AI tests code against the same understanding it used to write it, flawed code passes every test, so validation has to come from someone outside the build.

-Testing is still mostly manual - The knowledge of how a carrier's systems behave sits in the heads of long-tenured testers, and Alex says manual coverage at most carriers lands around 10 to 20%.

-The digital blueprint - Emtech maps a carrier's systems from the e-app through underwriting and policy admin to the issued policy, which Alex says adds up to hundreds of millions of test cases.

-No AI in the testing - Emtech builds the blueprint from deterministic data and runs on-site on desktops, because most carriers won't expose their back end to an outside AI.

-The enterprise AI licenses carriers aren't using - Alex argues that carrier AI is blind to the back end, and that giving it an accurate blueprint is what makes it useful, from user guides to product filings.

-Regression across legacy and acquired systems - End-to-end regression covers mainframes, acquisitions and issued PDFs, and runs in parallel on testers' desktops in days instead of months.

-The future QA team - Alex expects testers to become the stewards of AI inside carriers, because they are the people who understand the systems end to end.

Notable Quotes

"We need technology to test technology, but it cannot be the same technology."

"The knowledge that people have in their heads is often not documented."

"If you take that and you give that to the enterprise AI, that's the missing piece."

"Testers need to become the stewards of AI enablement within organizations because they're the subject matter experts really understanding end-to-end."

Resources

Guest:

Emtech Group: https://emtechgroup.com/

The Book on Testing: https://books.friesenpress.com/store/title/119734000184141596/Alex-Rodov-The-Book-on-Testing

Alex Rodov on LinkedIn: https://www.linkedin.com/in/alexrodov/

Host & Organization:

Joshua R. Hollander on LinkedIn: https://www.linkedin.com/in/joshuarhollander/

Horton International (USA): https://www.horton-usa.com/

Insurtech Leadership Podcast (LinkedIn Showcase): https://www.linkedin.com/showcase/insurtech-leadership-show

Subscribe & Review

If you enjoyed this episode, subscribe on your favorite platform and leave a review. The Insurtech Leadership Podcast is available on YouTube, Podbean, Apple Podcasts, and Spotify.