François Scharffe: An Agentic Harness for Knowledge Engineers – Episode 58

François Scharffe
As AI unleashes a host of new tools and agents to help build knowledge graphs, engineers like François Scharffe are tasked with harnessing these new capabilities and deploying them effectively.
François brings decades of AI experience to the goal he set for himself earlier this year: to automate his own job. He's making good progress, but he's also finding that there's still plenty of need for him and other humans in the workflows he's designing.
We talked about:
the upcoming 2027 edition of the Knowledge Graph Conference
his long history of working with various kinds of AI
the distinctions between knowledge engineering and software engineering
the rationales behind his choice of Pi as his coding harness
an overview of his harness set-up and knowledge engineering workflow
the benefits to knowledge graph building that AI brings
the variety of tools and agents he has built for tasks like requirements specification, data sourcing, ontology design, term extraction and re-use, entity resolution, data mappings, ETL pipelines, etc.
the evaluation processes he's using to assess the benefits of his new workflows
the ongoing need for humans in the loop, particularly for tasks like ontology creation
how he has adapted his work style for the AI era, in particular the number of tasks he now does in parallel with AI
how he manages ongoing improvements to his system - tracking decision traces and implementing introspection steps as well as devising scenarios and using "critic" agents to improve processes
the reason he downplays prompt writing and review in his operation in favor of system evaluations
his scientific approach to system evaluation, aiming for reproducibility when moving from one model to another
the focus on agent memory in the 2027 Knowledge Graph Conference program
François' bio
Francois Scharffe is a hands-on enterprise knowledge architect and semantic AI leader. He has built and stewarded production knowledge graphs across several domains, designed ontology, taxonomy, metadata, entity resolution and data integration capabilities, and connected structured knowledge systems with AI and agentic workflows. His career spans enterprise engineering, data and AI strategy consulting, product leadership, research, and community building. François is also a founder of the Knowledge Graph Conference, a leading forum for knowledge-centric AI technologies.
Connect with François online
LinkedIn
email: francois at knowledgegraph dot tech
Resources mentioned in the interview
Pi coding harness
the original xkcd "compiling" cartoon
the cartoon updated for the AI era
Video
Here’s the video version of our conversation:
Podcast intro transcript
This is the Knowledge Graph Insights podcast, episode number 58. AI unleashes a whole host of new tools and agents to help engineers build knowledge graphs. Harnessing these new capabilities and deploying them effectively is the job of the modern knowledge engineer. François Scharffe brings decades of AI experience to the goal he set for himself earlier this year: to automate his own job. He's making good progress, but he's also finding that there's still plenty of need for him and other humans in the workflows he's designing.
Interview transcript
Larry:
All right. Hi, everyone. Welcome to episode number 58 of the Knowledge Graph Insights podcast. I'm really excited to welcome back François Scharffe. He was in the first episode of the podcast series a couple of years ago with his colleague Tom Deely. They run together the Knowledge Graph Conference. But François is also a practicing in-the-trenches knowledge engineer in New York City, and we're going to talk today about some cool workflow and architectural stuff he's developed. But welcome, François. Tell the folks a little bit more about what you're up to these days.
François:
Hey, Larry. Great to be with you today. What am I up to today? A bunch of things. I mean, the number one thing is the Knowledge Graph Conference, which we're preparing for May in New York. So we're moving from Cornell Tech to Manhattan, Central Manhattan, Bryant Park. This is exciting, larger venues, more rooms, larger rooms. And we're going to open the program, the call for presentation soon. So if you're interested to discuss knowledge graphs, you have interesting projects to discuss, then yeah, keep an eye open for the call that's going to come out soon.
François:
Yeah, but I think you invited me to talk more about my other activities around knowledge graphs. As you say, I'm a knowledge engineer. I started my career in academia, and I've always come back to it at some point, either teaching or doing research. But now for several years I've been engineering, so I've been a knowledge engineer, in the weeds of the practice, and recently got pretty excited to see how AI is changing the job. So happy to discuss that if you're interested as well.
Larry:
Yeah, no. We had a quick chat. I was in New York last week, and we had a quick chat about it, and I just got unreasonably excited about what you're up to because everybody's wrestling with this stuff now, like how to incorporate LLM, generative AI and other AI tech into their engineering workflows of all kinds. But specifically for this audience, for a knowledge engineer to be doing the stuff that you're doing...
Larry:
Well, I guess, first tell me a little bit like... I guess like any software engineer these days, you feel obligated to use these tools. And then you have to harness them in some way, and that's led to this new term, I think arose within the last year, of the harness. And you're working on a very specific kind of harness specifically for knowledge engineers. Can you talk a little bit what the impetus was for that, why you decided to approach your work this way, and then maybe just a quick overview of how it works?
François:
Yeah. Yeah, yeah. So, I mean, the first thing I want to say is I'm a knowledge engineer, but also a background in general machine learning and data science, natural language processing. And I've been following the evolution of neural networks to language models and large language models for years, being more or less involved practically with them, but close to the developments there. And remember GPT-2 coming out and the poems it was writing about unicorns and finding that so amazing. But how long ago was it? I can't even remember. 2018, 2019.
Larry:
I was going to say that's about eight or 10 years. I remember.
François:
Right.
Larry:
Yeah.
François:
So when there started to be a big excitement about AI, I was originally quite skeptical, still following but not really being so enthusiastic as to its possibilities to do everything, especially in my field, knowledge engineering, that... We can come back on the details of it, but it gets pretty complex in terms of the sort of workflows that are involved. But yeah, last December there was this realization that... I think it was Claude 4.8 or GPT 4.8. Sorry.
Larry:
I think it was Claude. If I remember right, all my engineer friends got excited about Claude, I think is the one that really like, "Okay, here we go."
François:
Right. I think it was Claude. Yeah. And I was like, "All right. I think something has changed now. We passed a capability level, and let's go and jump to that." And so now being a knowledge engineer and being a software engineer, those are separate jobs. We're pretty clear about it. And I think the models and the software coding agents, like Claude Code or Codex or others, they have large prompts that makes them specialize for coding, so for coding tasks, for code reviews, for code basis maintenance, et cetera, et cetera. And that told me that those were not necessarily the best tools for building an AI that does knowledge engineering.
François:
Because one reason is that this will take a lot of context space with specific instructions for coding that are not necessarily relevant for knowledge engineering. Although there is coding involved in knowledge engineering, it may not be as complex. You will write scripts or some programs, but it will be always in a part of a certain workflow in a repeated and self-contained way compared to general software engineering. So that was one thing.
François:
And then another was we need specific tools and specific workflows for knowledge engineering. But also, I think there was part of me that likes open source and building my own things and adopting Claude Code was for me like, "This is something we have no control over. We don't even know what's in it." Now we know a little bit better, but at the time.
François:
So I was looking around and found this minimal coding harness called Pi. And there are others. There's OpenCode, but I went for Pi. It's very minimal and extensible and made for being extensible, for being extended. And so I went there and adopted it and was like, "Hey, I'm now an experienced professional, and I've been doing that job for years, and I have a lot of knowledge on what a knowledge engineer is. Why don't I try to build an artificial knowledge engineer to automate my job?" And the New York came, and the New York... I woke up, and I was like, "All right. This year, I'm going to automate my job. I'm going to make myself useless." So yeah, that was the goal with it.
François:
And yeah, harness, to go back to your question... So a harness is a minimal environment that sort of helps contain your wild LLM, your wide AI models. If you just talk to that model, interact directly with the model and asks it to do some knowledge engineering task, it will go left, it will go right, it will go where it decides to go like a wild animal, a wild horse, but not necessarily bring you from A to B in a straight line. Eventually you may reach your point, your destination,...