David: All right, letâs go ahead and get this fired off. Good morning, good afternoon, good evening, and welcome to the Rhize Up podcast. My name is David Schultz, and today, weâre going to be talking about the Unified Namespace, or UNS for short. I put this together because there still seems to be some confusion. We think we know what it is, but weâre not sure exactly what it is.
So, the intent here is to try to socialize and define a Unified Namespace. As I said earlier, I spent about three hours with some people over the weekend, and we were just hashing it out.
WHAT IS UNIFIED NAMESPACE?
David: Where we landed is the Unified Namespace is an approach for an event-driven architecture that includes a Pub/Sub technology like a broker and some sort of DataOps tools that allow us to create data models and publish and subscribe these data models through some kind of defined, topic structure.
Thatâs where we landed. Hopefully, by the end of this podcast, weâll be able to gel around a clearer definition. But with that, I am now joined by three very well-known people within the industry. I would call them the experts within Unified Namespace. They all have different approaches to it.
So, letâs go around the room real fast and make an introduction. How would you do that if you could just briefly describe a Unified Namespace when someone asks you what it is? So, Aron, weâll go ahead and start off with you.
Aron: I think you did a great job, David. I think Iâll just steal yours. So, Iâm Aron Semle, CTO at HighByte. My background is in industrial manufacturing for the last 15 years. I started at Kepler back in 2010. Then PTC, and now on to HighByte. So if I had to define Unified Namespace, I think the definition you gave right now, David, is probably a really good report by exception, real-time data, and single source of truth.
And I think the other folks will get into that. Itâs nuanced, right? Where the confusion is, itâs part of the magic of it. And then the problem with it. But looking forward to the discussion.
David: Yeah, absolutely. Thank you. Welcome, Aron. Jeremy?
Jeremy: Yeah, Iâm Jeremy Theocharis, cofounder of United Manufacturing Hub. Because most of our customers are large enterprises, when we go in with United Manufacturing Hub, we need to convince hundreds of people about the Unified Namespace. So, what we like to do is make things very, very boring.
By boring, I mean Unified Namespace is actually nothing really exciting. Itâs just IT best practices applied. So what is it for us? Itâs an event-driven architecture with message brokers. Very simple. Very well known. So what is it not? Itâs, therefore, not a database because itâs not real-time. Itâs also not point-to-point.
But people typically understand with Unified Namespace that, at the core, itâs actually something very boring. A lot of people and industries already do it, but they just donât call it Unified Namespace.
David: Sure, absolutely. And Kudzai.
Kudzai: Cool. Thank you, David, and thanks, everyone. So yeah, my name is Kudzai Maditereza, Iâm a developer advocate for HiveMQ. I do a lot of evangelizing for the Unified Namespace. Iâm also a founder of Industry40tv, a media education company running a popular YouTube channel and podcast.
Unified Namespace is quite a nuanced topic but fundamentally is about the event-driven architecture behind OT/IT data integration. But a lot of times, I find that explaining it in mostly technical terms loses some of the audience.
So I think itâs more apt to talk about Unified Namespace, at least in my case, as being that data platform for continuous innovation. And I think weâll get a little deeper into that. For me, itâs the platform that gives real-time access to all data from all parts of your enterprise OT and IT, allowing you to continuously innovate on top of that data without having gone through a lot of friction around it.
So yeah, excited to be here and looking forward to exploring it some more.
David: Excellent. Well, itâs great to have you here as well.
And so I defined it as an event-driven architecture. There is a broker, and there are some DataOps in there. So Kudzai, could you lead us off by talking a little bit about the brokerâs role in this whole process? Why is Pub/Sub so important to a Unified Namespace?
Kudzai: Okay, to set the context, itâs important to talk about how we arrived at this idea of a Unified Namespace. Right? And what are the challenges, really, that weâre trying to solve there? I think that will help us paint a good picture of the idea of Pub/Sub and the broker.
Typically, a company has this idea of becoming a data-driven company because that helps address many different data use cases that you might want to address to maintain a competitive advantage. So whether itâs existing energy efficiency, OEE, or advanced analytics use cases you want to address, a Unified Namespace is something that, for you to address all these use cases, you need to connect it directly to all the different systems on the shop floor and to IT. So, youâve got this OT/IT integration whereby youâve got apps that are connected directly to equipment. So, each time you need to address a certain use case, you need to set up that connectivity infrastructure. Thereâs a cost related to that. And thereâs repetition also involved there. The time it takes for you to bring up, especially from different sources, up to IT, it takes a lot of time, months and even years.
The idea of the Unified Namespace is to say youâve created that hub of information where you have all these connections already set up, and youâve got all this information already being pushed to that central repository of information that gives you that real-time snapshot of all the, the current events and state of your business without having to worry about the underlying, connections. So, this idea that you are pushing data from the edge into this central repository of information and also from IT. Itâs information where you have this common format data unified and that is accessible through that single interface.
So, this is where the broker comes in. It acts as the platform that allows components or systems to connect from the edge and push the hub of information that then gives access to real-time events. And what really makes MQTT or the broker ideal for this scenario is that you donât want to just push data using it as if you are using a legacy protocol, where you just like saying the topic is say, a tech name, right? You want to be able to organize the data in a way that can be discovered semantically by anyone who understands the organization and the structure of the organization.
So MQTT gives you the ability to be able to organize information using its MQTT topic namespace. This is the part that the MQTT broker plays in the Unified Namespace. Itâs the source of real-time access to the status through a single interface without you having to connect to a thousand different servers on the shop floor. By connecting to one broker endpoint, you get access to all the unified, normalized data in a way that has already been organized and is being pushed using published communication.
I donât want to get in much depth about what Pub/Sub is, but at a high level, itâs the idea of the brokerâs role in a Unified Namespace architecture. So you still, at the end of the day, need some components that help get it out. And I think this is where Aron will be able to expand a bit on how you get that data out and then how you marry OT data to IT data within an MQTT information hub.
David: Before we move on to Aron, I want to make sure that I understand the brokerâs role. Itâs around this event-driven piece. But I need an easy way for edge devices or other systems that are going to instantiate that connection. So thatâs one of the first pieces of it is edge data can connect to something rather than having maybe a server client as the technology that I have to go and find all these. It enables easy access for all that data to get there. Then of course, now itâs that whenever thereâs new information thatâs part of the event-driven architecture as new information is available, I donât have to ask you for it. The Pub/Sub enables that new informationâyou just send it to me when you have it, and Iâll be sure that it gets out there. Is that a pretty good synopsis of the role of the broker here?
Kudzai: Yes, it is. And also maybe just to expand on that a bit, this idea of event-driven. As you mentioned you donât get to a situation whereby applications are reaching down to this job flow to ask for this data. Instead, it is pushed from the edge into the data infrastructure. What that allows you to do is create this idea of a single source of truth because all of the data that youâre currently viewing is the data that is actually being pushed from the edge, from the source of the data.
You can easily build your business model around it because itâs not data that has been aggregated through different layers. Itâs coming straight from the device at the edge into the data infrastructure. And, also the idea that you donât have an application that is pulling to get that information, it means that whenever thereâs a network failure with Pub/Sub, the edge component or the device is able to buffer that information and retransmit whenever that connection is restored, which in any other scenario, youâd lose that information whenever thereâs a break in connection.
So this is the importance of having that broker or Pub/Sub architecture in a Unified Namespace.
David: All right. Excellent. Yeah, I certainly donât want to overlook the importance of the ability to buffer data to ensure weâre getting everything there. So, I guess in that case, make sure that when youâre choosing a broker, you choose something that has that ability and all the technology supported.
So, Kudzai, you mentioned that as you were talking. Oh, go ahead, Jeremy, you had something.
Jeremy: This is more of a scientific explanation. We tend to go a little bit more theoretical. One of our most popular blog articles, we actually compared different MQTT brokers through it. We only got two cease and desist letters for that article, but itâs still online in its original form.
In the introduction, we try to give a more scientific reason why you need the message broker in industrial IT systems or, in general, in a system where you process a lot of data. You always have multiple building blocks. You cannot do everything in one application. And in order to connect those building blocks, thinking top-down, there are only three options on how you can do it.
You can send it through databases. Thatâs the first thing. So we have all the PLCs. They all send data in the database. Then you could do some batch processing on it. But as soon as it shows us that architecture, with a standard storing approach, you cannot do anything with real-time data anymore. All this stream processing. So data comes in and you change the color of a traffic light. Itâs not possible anymore.
The other way you can connect those building blocks is in service calls. So point-to-point. So you do a lot of point-to-point connections, and itâs actually a fairly reasonable approach. However, in manufacturing, a lot of these blocks tend to stay there for like ten years. At Google or Netflix, they can do it with point-to-point connections, but they also rewrite their whole applications every three years. I mean with factory stuff tends to be like this, then you have these spaghetti diagrams because you would never refactor.
And then thereâs only a third option, which is called asynchronous message passing. Itâs basically a message broker, which, unfortunately, adds a new component that can fail, which is bad. Then, the other side allows everyone to start talking to the message broker to exchange information in real-time. And then you put it in the database. So everything has advantages and disadvantages, but then manufacturing where the other two choices are mostly not a good way to go, a message broker is the only viable option left. Itâs usually a very good option.
David: It always helps when other people provide their own input. Certainly, a picture speaks more than a thousand words. It really helps me understand and unpack this, which is my favorite part of doing podcasts. I learn so much when Iâm on it. So, again, I canât thank everybody for their time and participation.
As Kudzai was talking, one of the things that came up was that we want to have these semantic data models. We need to publish them to a defined topic name structure. Because if Iâm just a device, and we all just start willy nilly doing things, I donât know that weâll ever be able to leverage the value of this Unified Namespace.
Aron, I think itâs HighByte that might have even pioneered the term DataOps. So, when I talk to people about HighByte and what it does, I say itâs a DataOps tool. I even mentioned that in my definition. So can you share with us what exactly is a DataOps tool? What is HighByte doing and why is it so critical to the architecture and design of a Unified Namespace?
DATAOPS AND THE UNS
Aron: Yeah. Sure thing, David. I think we started back in 2018. I think we looked back and did a Google search on industrial DataOps back then, and there were 70 results. And I think now thereâs 300,000. So we were talking about it pretty early. And obviously, thatâs changed over time. But still, itâs a pretty consistent message.
When we think about DataOps in industry, the classic example of a use case I like to give is a large industrial enterprise. I might have 50 facilities. I should be able to have a team of half a dozen folks that have the people that process the technology to manage my data movement from the edge to the cloud, in the cloud, back to the factory at scaleâwith the goal of driving the cost of change down to near zero.
So what does that mean? Hereâs a classic example. Some of our larger customers have AGVs in their factoriesâautonomous guided vehicles. So, forklifts drive around with nobody controlling them. A common problem with those is theyâll get stuck somewhere in the factory, and you need to be notified, and someone needs to go get them. Itâs kind of like your Roomba getting stuck under your kitchen chair, right? So thatâs a use case. Itâs a very practical one.
Letâs say you have 20 facilities that have AGVs across three or four vendors. What you do with the DataOps platform in that team of six folks is you can go in and say, âWhatâs the data we need from these AGVs to be able to detect when they get stuck?â
With that, you create a data model. And that data model is nothing more than maybe its location, last time it chirped data, maybe it was talking MQTT and a few other additional bits of information that say, âThis is the data we need from the AGV, regardless of the vendor, to be able to enable this use case.â
So, with DataOps, you can quickly enable that in one factory. Weâre not talking about months of integration. Weâre talking about weeks or less. You can then pipe that data to some system in the cloud. Maybe itâs going to Snowflake S3, maybe itâs going through MQTT as part of the UNS. Then, what you would do is youâd prove out that use case really quickly with the factory to say, okay, can we detect these AGVs getting stuck?
Once youâve done that, you should be able to take that use case, that data model and the technology you use to push that data in, replicate it across all of your factoriesâall 20âin less than a month, and enable that use case.
At the same time, you might find that you tried that use case, but it delivered no value. You just throw it away. So itâs DataOps of the concept of Iâm going to create this process, people and technology that allows me to test these use cases really quick and then either scale them up quickly or reduce them. This is part of the UNS story, too. Once you make this data readily available, people will start to use it, and then theyâll want changes to that data.
Theyâll say, âHey, we donât need these attributes in the data payload, but could we have these, or could we scan at a faster level?â The key to DataOps is ensuring people, process and the technology can make those changes at scale, at near zero cost. You should be able to add an additional attribute of data, wire that up across all your facilities, and publish that to your end system with very little work.
Thatâs the core of DataOps. We think about DataOps in terms of UNS, the part that applies directly to UNS, is that data model, that semantic model, like you said. If I have every PLC just chirping raw data into the UNS, itâs not very usable. Traditionally, factories donât have this concept of a nice rich namespace with a logical way we think of the factory. And then that data mapped to the actual data in the PLCs. MQTT and DataOps is a way to model that data at the edge to create those models and then move it semantically into MQTT. In this example, in terms of UNS, in a way that a human could come in and look at and say, âThis makes sense. I can see this site area line, this is the machine, this is the temperature sensor, regardless of the vendor.â
It abstracts away the details and the mess of the different implementations and technologies at the edge, so by the time it gets into the broker, it makes logical sense.
David: Excellent. So if I understand, we now have this broker. Weâre going to use MQTT in our example here for this Unified Namespace because it is fairly common. I have client instantiated data. I can update data as thereâs new information available. And I can also buffer that if for some reason thereâs a loss of connectivity thatâs there.
The DataOps piece comes into whatâs sometimes called data governance. Thereâs going to be a data contract between the producers and the consumers of the data. The âcontractâ will say Iâm not just going to willy-nilly publish raw data, Iâm going to create these agreed-upon, semantic data models. I always use a pump or a compressor, some type of asset to say this is what that payload is going to look like. And then Iâm also going to publish it to a topic structure that has already been defined.
So, commonly, weâll use the master data model. Thereâs a lot of references to ISA 95, using that hierarchy of enterprise site, area, line, cell. So we want to publish data thatâs there. Thatâs the piece that DataOps takes care of. We now have an enabling technology, but letâs make sure that weâre sending data in a way that everybody can understand. And if I needed to go find data, I would know where to get it. Does that make sense?
Aron: Yeah, exactly. And I think DataOps and UNS overlap quite a bit. DataOps is the flexibility to model and move that data wherever you want. MQTT happens to be a really easy, useful way inside the factory and even out for multiple subscribers to come in and access that data. So, it kind of gives you that semantic ability but also the flexibility to move it wherever it needs to go.
David: So, continuing on with many of the conversations around the UNS, one of the first questions that almost everybody asks early on is, well, is there a database? Where do I store my data? Because if itâs just fine publishing to it, does that mean that the UNS is a database? Actually, no, itâs just the current state of the business is the way I understand that. But maybe, Jeremy, you can help us understand a little bit about, does a UNS store data, or if it doesnât, then how do you go back in and retrieve some of that data?
Jeremy: Yeah. This is also something like our scientific approach to the message broker, and it always sounds a little bit unnecessary, but in situations like this, it really helps bring the discussion forward. So, I already said, there are only three ways we can connect building blocks. And if you do it by your database, then you donât have this real-time.
So, we already established the Unified Namespace in its core is not a database. It has a message broker. But then there comes the questions: With all the real-time data, I want to calculate OEE, I want to see the story. Where is the story in it? How would it work?
You have a Unified Namespace or a message broker. You have this DataOps or the contextualization layer, which helps you model everything, and then you have an additional database, so that whatever you have in the Unified Namespace, you can actually also put it in a database.
And there I think you can go deep down into rabbit holes, because what do you want to store? Because you can send a lot of information through the Unified Namespace. I would say the easiest thing is time series data. So you have it, then you send it through, and then you can define a certain schema if data is in a certain type of format. For example, if we have this topic, it will then light up in a time series database. And you can apply the same for other things. For example, if you have more of an ISA-95 style of informationâlike work orders, new product etc.âyou could store it in a more relational database.
Some people in the community try to combine it all. They say, âLetâs also put the database in there and letâs try to mix it up and put a graph database in there.â
From our experience, it just makes communication very hard. So, thereâs a message broker for real-time data, a database, and a microservice component that could also be the DataOps layer or something else. That component subscribes to the message broker to certain types of topics and then stores the information in the database.
And this is how weâve seen most companies approach it. Of course the exact tools changeâtype of databases, message broker. But with this approach, we aim to give it to any IT person, so that they will look at it and say, âokay, yeah, this is boring.â
Then, there is this step between what if I want to store data, but not permanently, only buffer it for a short period of time. For example, during connection outages this gets a little bit tricky because you donât want to use a database because theyâre only done permanently, but also MQTT, Iâve not seen actually broken yet with like a proper queue system in there.
So we use, for example, Apache Kafka for this as an addition. You could also do it in your own microservice that buffers the data in between. But these are the main considerations. Now, we could go into all types of details. I could talk for hours about this.
David: Thank you for that. So if I understand, weâre just limiting the Unified Namespace, and how we are defining it. Thereâs no persistent data or data storage. Itâs purely just for the Pub/Sub functionality, using a DataOps tool to make sure that weâre bringing in semantic data models to a defined topic structure. And then, depending on how you want to query or retrieve that data, that becomes more of a function of what youâre trying to do. Did I understand that right?
Jeremy: I donât know about the last part. Can you rephrase that?
David: Yeah. So now the endpoint depends on how I want to consume that data or query that data. Thatâs not something Iâm necessarily going to retrieve from the Unified Namespace. Thereâs going to be a database or some other piece of technology. Thatâs where I go to retrieve the data. Itâs just that everything has traversed that, and thereâs an agreement on the data Iâm going to find. Is that right?
Jeremy: Yeah, exactly. And what makes it really difficult is to ensure a consistent data model across everything. Iâm still writing an article on this thatâs half-published. It takes a lot of time because now you have data models in the PLCs itself. Then you have a data model in the UNS, and then you have the data model in the database. And you want people to still be on that set of the PLC, to still understand whatever happens to the database. So this is something that is really challenging, to design your data model so it works across all three of those components.
David: Yeah, I think thatâs a very fair statement. And actually, one of the things that I wanted to talk to Aron about is the ideals as weâre building our data models. This is very common in programming. We have this concept of inheritance versus composition.
INHERITANCE VS. COMPETITION IN DATA MODELS
David: When Iâm thinking of creating these data models, I might be inheriting something where Iâm a member. So Iâll go back to my easy examples, like the pump. I might just have a basic pump model, but some of the assets in my equipment, some of the pumps, now have more information available on them.
Do I want them to be a child of a pump? Or should I just create two completely separate independent models of this type of data? Or do I start bringing other data models and nesting them within an overall model? Suddenly, it can be pretty cumbersome and complicated on top of the three use cases you already mentioned, Jeremy. So, from a DataOps standpoint, Aron, how do you generally approach the data modeling and all these various scenarios?
Aron: Yeah. I look at those two extremes. You could start really simply and just model the bare minimum of what you need. We see some customers do that. We think they probably have more success being use-case-driven. As I mentioned, the AGV case. Start there. Go. Learn. Adapt.
Youâll have other customers that will go to the other extreme. I would call that digital twin, right, where itâs almost academic. Theyâll spend six months trying to model the factory perfectly, using UML on a bunch of stuff, and then trying to break that back into semantic models. Theyâll end up with the perfect model. Theyâll go to implement it and find out itâs really difficult. And hey, the end systems canât use this anyway because it has too much hierarchy, or we messed up here.
The sweet spot is probably somewhere in the middle. In terms of inheritance versus composition, we have a whole modeling guidebook. I think we wrote it a year or two ago, and itâs actually really useful. So I encourage people to go look at that. Thereâs a lot of practical advice there in terms of modeling, which is new to the space.
But in terms of inheritance and composition, with inheritance, we tend to lean towards composition more. And this is kind of academic as well. But the thing with inheritance is that you kind of create a hierarchy as a result of inheritance.
As you mentioned, David, you have a base model of a pump, which you derive for the Mitsubishi pump, and then different pump types. For the audience, you think of inheritance; you think itâs a relationship. So I have a car thatâs my base model car, and then I have a Volvo and a Subaru. I start to add attributes off of those base models and build that up. You end up with this implicit hierarchy thatâs somewhat coupled. What you end up doing is, if you go with the inheritance route, later down the road, you have to go change some base models. And there might be implications in that across all of your model definitions that you didnât quite anticipate. That can make changes more difficult.
So like the classic example, if folks are familiar with OSIsoft, now owned by AVEVA PI System. They have a product called Asset Framework, which is over ten years old now which is kind of a pioneer in modeling historical data on a story. And if youâre familiar with their system, they use templates, which are models, and they use inheritance. And if youâve ever dealt with a really large PI AF system, youâll instantly be able to tell why inheritance gets tricky. That coupling starts to make things really difficult.
So, composition, on the flip side, is a little more flexible. It just says, hey, I have a model, and I want to inject in that model a set of attributes that are, letâs say asset metadata. So, the name of that manufacturer, the asset, when it was last serviced, the year it was installed, that kind of thing. You can place that in any of your model definitions that are an asset, change that set, and have a smaller change set across all of your model definitions. Because only the things that are composed of that will change. So it gives you some more flexibility.
Composition is probably the better way to go, but itâs a great question. I would also say donât go digital twin. Digital twins work really well in building automation because buildings donât change. Theyâre built once, and thereâs a code and everything set up.
When you look at discrete process or batch manufacturing process, maybe not so much. But with a lot of data, things are in constant flux. If you try to design your models around digital twin to perfectly replicate that physical asset digitally, youâre just going to do a lot of work. And you might only leverage 20% of the value of that work at the end of the day, given the data you use.
David: Excellent. Iâve always liked the idea of inheritance because, from a data governance standpoint, it means I make one change, and I know thatâs going to proliferate or federate across the entire system, versus the composition where I might have to go in and do some other activities.
But I hear what youâre saying. Thereâs a lot more flexibility. I tend to be more on the composition side myself. It just means part of our data governance is when there is an update, weâll call it a best practice that everybody involved in this DataOps tool is responsible for updating their assets accordingly. Is that a good approach to handling that?
Aron: Yeah. And another practical example. You have an injection molding machine that has two motors, and then you have one that has three. You could do that inheritance, but doing that model via composition is a little easier. The flip side of that is again, you might not want that level of detail. By the time you get up to your cloud system and the AI and ML, because if Iâm coding against the data youâre providing, and Iâve got to deal with this one has two motors and three motors and five motors, I might not care. That just creates complexity for me.
So you always have to think about that end-consumer in your use case of the data and provide just enough thatâs needed to simplify their lives. So, pros and cons.
CHOREOGRAPHY VS. ORCHESTRATION OF DATA
David: Yeah. Very good. Great discussion. Thank you, Aron. So, continuing on the same idea, thereâs another concept thatâs known as choreography versus orchestration.
Moving this data around, how do we want to ensure that the right data ends up at the right place? So, the concept here is choreography. If youâre watching a play or an actual dance group, the choreography is Iâm going to do something based on what somebody else is doing, but nobodyâs necessarily controlling me or telling me what to do. I have to be paying attention to where everybody else is so I know where Iâm supposed to be and what Iâm trying to do. So itâs very independent versus orchestration. Think when you go to the symphony or the orchestra, the conductor is always providing that guidance. And I know Iâm not going to play my instrument until the conductor has looked at me and said, okay, now itâs time for you to play. And thereâs a lot more control over that.
The benefit of choreography, of course, is that all my systems can act independently. Thereâs a lot less overhead thatâs associated with that. Versus orchestration where now you have a system that is telling everything what to do, and it has to manage everything. And thereâs a lot that goes into that. In this conversation about orchestration versus choreography, Jeremy, what are your thoughts on that? How do you approach it? What do you think works? What should you try to avoid?
Jeremy: Yeah. I think we can approach this topic from multiple perspectives. Let me start with one, and then you can ask a follow-up question.
The first point is that on the topic of orchestration versus choreography, you could also rephrase it to something like central governance versus edge-driven. I would even say that orchestration, the central governance, is actually a failure mode for industrial IoT for big projects because this is how IT tends to think. Typically, in IT, problems are a little bit different. You have one cloud and one server; everything is very close. So, what you can do is you can have one central Git repo, and one central database. And from this, everything else builds.
I would say itâs a failure mode in manufacturing because in manufacturing, you are working with distributed systems. So what does it mean? You cannot assume thereâs always going to be an internet connection. The connections between all the components are often very unstable. You donât want your production to stand still if the internet connection goes down. Also, from a security perspective, you have all these demilitarized zones, and every layer acts independently from other layers. Of course, if there are longer outages, letâs think about the PLC. If there are longer outages, no information comes in from the SCADA system or the MES system. So it would probably stop after an hour, but it could work independently for a certain time frame.
Manufacturing itself is very much edge-driven. So itâs about choreography. There are a lot of different perspectives. The result of edge architecture is that itâs getting quite complex. If youâre designing a system and you cannot apply these typical cloud IT principles, and you start doing it, you might not be prepared for the result. Because how does it look then if you start with choreography, how does it end up? You have a lot of edge devices everywhere. There could be multiple message brokers. People in IT often ask, âWhy would you have multiple message brokers? Just have one.â No, we need every component to act independently from the others, so we will have a message broker at least four per side, if not even per production line.
Then you need to sync between it. And now, you can encounter a lot of different problems that arise from this architecture. I mean, the first thing is you could argue, okay, maybe letâs streamline everything. Maybe you can comment on that. I think itâs impossible to streamline that in manufacturing because everything is built to be edge-driven. So, if you want to work with it, youâre going into distributed systems. And there are a lot of challenges that arise that a lot of people also ignore. But as soon as they start to scale out, they realize this.
One example of data flow is what type of data youâre sending through the system. So now imagine we have a message broker for each production line and one for the site, and we have one in the cloud to gather all the data. If we know the multiple cases you have, for example, if you have time series data, itâs really important that 99.9% of the data arrives in the cloud.
If there is an internet outage, itâs important that the data is buffered so that it never gets lost and stored on disk. But to be honest, itâs acceptable if sometimes data points get lost. So if you send a data point every second, and then it gets lost once, okay. But to be honest, I think a lot of people have this problem, but they just simply donât notice because itâs fine.
But then, when youâre talking about things like adding a new order or sending commands, for example, in the cloud, you have a button to start a machine. Then things start to get complicated because how should the system behave if thereâs no internet connection? You donât want this command to get stuck somewhere and then execute hours later, so this is what makes choreography so difficult.
We try to build a product that gives you the overview over that. So you have all of these instances, you have all of these edge devices. We give you a single view that allows you to see the health of each of your nodes, the health of each of your network devices. So in case something goes wrong, you are notified about it. And you donât have to go through hundreds of VPN tunnels to get to that edge device; you just need to check that the problem is somewhere else. But, now Iâm stopping my monologue and getting over to you, David, for any follow-up questions. I hope this was what you asked with choreography versus orchestration.
David: Thank you for that. I tried to pick words that were going to be very difficult for you to say. And Iâm glad I was successful with that.
So, your comment about orchestration versus choreography is that you were talking about the pitfalls, especially if you do a full-blown orchestration where the command is central to that governance.
I wondered if maybe we could come back to the Goldilocks principle. Can there be a balance? There will be some choreographed systems, but other systems are more orchestrated because we want to have more control over that data and understand whatâs going on. Would that be a good approach? Maybe we could try to keep it as choreographed as possible but add orchestration as needed.
Jeremy: Yeah. So what we try is to bring both worlds together because, at the core, it needs to be choreography. Everything needs to work independently of each other. But on the other side, just from a maintenance perspective, you donât want to have all this chaos. You want to have central view. You want something like Git and GitOps to configure it. So you need to meet in the middle. Our current hypothesis is the data infrastructure needs to be distributed. But then the configuration of that can still be centralized. So that would be the current hypothesis on where the trade off needs to be.
David: All right. Excellent. Thank you for the follow up on that.
So, another question that we get asked a lot, and I think Kudzai, you can speak to this, is about some of this conversation around inheritance and composition or orchestration versus choreography.
QUALITY OF SERVICE WITHIN MQTT
David: The conversation is all about needing to know that my message made it to where it was intended to be sent. While weâre not here to design a point-to-point per se, I want to know that if I have a change that is relevant to some other system, there needs to be some assurance that that message got there. What comes up oftentimes is, specifically within MQTT, this concept of your quality of service or your QOS.
So, while QOS2 will ensure that the message makes it there, my understanding is that it is really just the relationship between the client and the server, not necessarily from the client to the other client. So, can you just talk a little bit about the quality of service in general and its implications? And then, for those that must make it there overnight messages, what does that look like using QOS2 between two clients?
Kudzai: The same applies. Thatâs the same question that we often get from customersâthey want to send MQTT commands. And just before I go into the QOS, most of the time, we basically tell the customers that if you need to control your robot, like you need to perform some kind of closed-loop process control, you are better off using OPCUA or some other field-level protocol because it guarantees synchronized communication, where you get the response, the immediate feedback about whatâs going on. So, MQTT is really not built for sending those critical commands or messages.
As far as quality of service is concerned, yes, you are 100% right. The relationship between the quality of services between the client and the broker. With quality of service, the broker guarantees message delivery to the actual client. What that means is that when the publisher sends a message, it will get an acknowledgment that itâs been received by the broker and that the message is being sent to the client. However, what happens after the message is handed over to the client is not something that you get within the MQTT framework. So this is something that would typically be addressed in a direct, point-to-point protocol. However, with them out of service, there is the guarantee that even when the network fails during that exchange, all of the clients that did not receive that message during that network disconnection, the broker would follow up with all those clients and make sure that they do send an acknowledgment. Then that actual acknowledgment is sent back to the publisher or whoever is interested in knowing what acknowledgment has been sent. So, that information is not lost. Right?
So the quality of service guarantees you all of that information as long as youâre using persistent sessions. But again, if you want to control a robot or if you want to get data out of a database and search through what happened, you are better off connecting directly to that application and then that snapshot of information, that view, you could then publish it to the MQTT, Unified Namespace for all the other components that might be interested in that result. So, MQTT is only for real-time events instead of across your entire enterprise and not for transactional interactions.
Jeremy: To follow up on what Kudzai said, first of all, please believe me. I studied mechanical engineering, so I studied the vibrations caused by CNC machine milling. I realized that sometimes referring to scientific IT literature. And taking a look at, hey, how do these big companies solve it? That really helps.
I think this question of quality of service is interesting because it is already kind of like a soft problem. So what âquality of serviceâ means is that it guarantees the communication between the client and the broker that it happens only once. But still, it doesnât guarantee that through the system. For example, if you have one message broker that originates from one client, it actually ends up at the second client. Or even if you have multiple message brokers that actually end up through multiple brokers as a bridge to the cloud, whatever, it doesnât guarantee you that. So, you always need to solve this on the application layer.
Itâs not the problem of MQTT. Itâs just how IT protocols work. Even the underlying fundamental part, even in TCP, you have this concept of resending messages, but it only goes from the network stack of one device to the other network stack. So if anything gets lost there, theoretically, thereâs already a retry mechanism in there. Also with MQTT, every day, there is a retry mechanism, but still in MQTT thereâs the option that the message gets delivered somewhere. For example, it gets delivered to the client, but something goes wrong between the delivery of the message and the actual processing of it. How we typically solve that, and how we can send commands through the system is by giving the message a certain type of payload, to actually design the protocol and the application level to actually ensure that some of the problems donât arrive.
For example, you would have, like, a request and response mechanism even through MQTT. And if you really need to ensure that something was really executed, maybe we can link to the article further down. It really goes well into the details of exactly this quality of service discussion. So if anyone is having exactly these type of thoughts, I can definitely recommend going into this article here.
David: Yeah. You shared that article during our conversation about this, and I found it exceptionally helpful because it brings everything that weâre doing full circle. One of the questions and the struggles that I have had is that weâve always talked about this Unified Namespace both for the data and the events and transactions. And to me, these events and transactions are asynchronous. And itâs not like, say, time series data thatâs just constantly flowing in. As we talk about events, itâs either the car arrived somewhere, thatâs an event, or you could put an event frame aroundâwe had a production time or downtime or thereâs a shift. An event that could be time-bound is another aspect of it. And then, of course, there are transactions that are at the end or the beginning of those events. There are certain payloads that publish things off.
So, you know, maybe these are just last questions. In general, is a UNS really geared for time series data, or can it be used for events and transactions? If so, what is the best way to approach the events and transactions? So at this point, anybody can chime in on thoughts around that idea.
Jeremy: I think that the UNS is suited to some type of degree for events. Iâm still investigating this, but my gut feeling tells me the more complicated it is, the more itâs better to have it in a different type of system.
One example, a state machine, you know what to enforce, that the production is always in a reliable state. So that means a line PLC or a SCADA system. So if you want to use UNS as a communication through everything. So, send a machine command, and you can show with the techniques that I mentioned that data actually arrives there.
But who is taking care of the state of the other production line? So, what do I mean with state? When the machine is starting, in order to go from the âstartingâ state to the âstartedâ state, PLC must first respond with a certain type of command. There must be a reset. Whatever. This type of logic of events. Itâs much better suited in an external system because UNS cannot handle the state thing, but you can use the UNS to send this information through it.
I donât know if it makes sense. And maybe others can chime in when you have some more experience in this regard.
David: Yeah. I think itâs going to be a topic that certainly gets discussed on future podcasts. So, we are a little long on the podcast. I do apologize. I believe Kudzai, you need to head out real fast. Do you have any final closing thoughts? We started off with âletâs define a UNS.â Do you think weâve achieved what it is we set out to do here?
Kudzai: Thank you. Yeah, I think we did. So Iâll quickly go through some final thoughts about the Unified Namespace and the idea of events and transactions. Again, just to reiterate, MQTT is a protocol that gives access to the real-time events and state of your enterprise. Itâs not so much used for controlling robots or accessing the database. That needs to be handled outside of the Unified Namespace with a direct connection to the database or a robotic system using a field pass protocol or Rest API endpoint, whatever the case may be. So MQTT holds that current state of information, and using features such as retained message, it makes sure that youâve got Unified Namespace always living within your MQTT broker, not as a database, but just as that snapshot. So, whoever joins the network is able to discover what the current state of any component that is connected to the Unified Namespace is.
And if the network happens to disconnect on any one of the clients, that information persists, but not as a database. Again, just as a mechanism to make sure that that information is redistributed whenever these different components rejoin the network.
We spoke about the database connectivity aspect earlier. For example, for HiveMQ, we provide a connector that allows you to persist all of this Unified Namespace. So you could say, select different namespaces you want to consume where they connect, Iâm going to say Snowflake. If you want to sit in a namespace that you want to persist to, you use a connector that allows you to do that. If you want to persist data to a MongoDB, thereâs a connector that allows you to do that. Or if you just want to send it over to Kafka, thereâs a connector that allows you to do that. So MQTT itself doesnât persist, but it gives you a mechanism to be able to persist your data outside of the broker.
Lastly on the idea of orchestration and choreography. So I mostly like to think of it as it helps to look at Unified Namespace as a way of sharing data rather than collecting data. I think weâve been accustomed to the idea that data is a byproduct that merely needs to be collected. But itâs the mindset shift where you start to look at it as this idea of having to share data, which really encourages owners of that data to actually package it in a way that is usable on the other end because now it is being shared rather than just being made available for collection.
And itâs kind of like a mix compromise. We have central governance, and youâve got this distributed approach to it. So itâs a balance that needs to be struck.
David: Yeah. So it sounds like thereâs a lot that goes into a Unified Namespace. And the ways it can be approached and depending on the use case really will indicate how you go about it and actually developing the formal architecture to it.
So, Kudzai, if you need to drop, please do. I do appreciate your time. Thank you for that. But before we sign off, Aron, weâre talking a little bit about time series data, versus events and transactions. Any thoughts that youâd like to offer on the topic?
Aron: Yeah, as you say, hats off to Kudzai and Jeremy for their time. I think to say MQTT, donât try to do transactions through there. Donât try to do commands. I think it is a message we kind of need to deliver to the broader market. And itâs refreshing to see that because we are in the early days of UNS. Itâs even on the Discord channels and stuff nowâpeople trying to figure it out and itâs like, just donât do it. In most cases, itâs not the way to do those.
Whatâs interesting about the point-to-point connections is that you need a guarantee the data is going to be delivered to S3 or some other source, and youâre running it through MQTT, can you kind of hack it, using the client IDs and stuff to say, âhey, if this client drops, then start storing.â Yeah, you can do it. But it might be a sign you might want to send that data to an alternate path. Maybe that should go to MQTT, but it should have a direct connection to S3 or Blob to guarantee that delivery.
So what I think is interesting is UNS has caught fire. Itâs definitely here to stay. I think MQTT is a core component of that report by exception. But you know, all these conversations and confusion is really the market saying this is great, but what about this? What about this? And what about this? Because we have slightly different use cases in manufacturing, MQTT design for oil and gas, and the protocol itself doesnât encompass all of those.
And Kudzai was very honest about that. And I think thatâs great. I think MQTT is a core part of UNS. And I think the interesting part is how does UNS grow? How do us, as vendors and technologists, support our customers to enable these other use cases through best practices? I think, David, itâd be interesting to come back in six months and be like, okay, what did we say that was right? And what do we say thatâs now changed? I think it is moving that fast.
David: Oh, thatâs an interesting perspective. I hadnât thought about how we could revisit this. And what I said at the beginning of this call is, well, actually thatâs not what a UNS is. So, I think maybe the struggle is that weâre still aligning ourselves on what the technology is and whatâs in the approach because the intent is that we want to create these data-driven organizations. We want them to have access to all of that data in real-time, and they can use that for solving problems, whether itâs asset health or predicting the failure of a piece of equipment. And what does all that look like? So, before we sign off, Jeremy, any final thoughts you want to share with us?
Jeremy: Yeah, to add to that, the opinion I stated is clear. So, I said that it would be fine to send some comments to the UNS. But for more complex things, itâs probably not that well suited. But for controlling the production, you can still use UNS, but itâs not that intuitive to do. Just sending messages through it, thereâs a lot of complexity behind it. So if youâre unsure about whether to do this or not, I would tend to say probably for now, you can use it, but be careful of what youâre doing.
David: Excellent. Thank you. And really, Kudzai was talking of Industry40tv or the HighByte website, and certainly, with UMH and Jeremy, thereâs a lot of great content about things to think about as far as weâre doing this.
So, at the beginning, I define the UNS as itâs an approach to an event-driven architecture that utilizes a Pub/Sub technology like MQTT, along with a DataOps platform, that weâre creating semantic data models and publishing it to a defined topic structure. It seems like thereâs a lot of other considerations in terms of composition versus inheritance or orchestration versus choreography and taking advantage of some tools that are in there. Just make sure that what tool youâre using is the right tool for the job.
Well, thank you, everybody, for your time. I want to thank the other people who were involved in all these conversations as well because itâs certainly something that Iâm trying to get my head wrapped around. But thank you, everybody, for your time. I look forward to seeing you on our next podcast.
So, the intent here is to try to socialize and define a Unified Namespace. As I said earlier, I spent about three hours with some people over the weekend, and we were just hashing it out.
WHAT IS UNIFIED NAMESPACE?
David: Where we landed is the Unified Namespace is an approach for an event-driven architecture that includes a Pub/Sub technology like a broker and some sort of DataOps tools that allow us to create data models and publish and subscribe these data models through some kind of defined, topic structure.
Thatâs where we landed. Hopefully, by the end of this podcast, weâll be able to gel around a clearer definition. But with that, I am now joined by three very well-known people within the industry. I would call them the experts within Unified Namespace. They all have different approaches to it.
So, letâs go around the room real fast and make an introduction. How would you do that if you could just briefly describe a Unified Namespace when someone asks you what it is? So, Aron, weâll go ahead and start off with you.
Aron: I think you did a great job, David. I think Iâll just steal yours. So, Iâm Aron Semle, CTO at HighByte. My background is in industrial manufacturing for the last 15 years. I started at Kepler back in 2010. Then PTC, and now on to HighByte. So if I had to define Unified Namespace, I think the definition you gave right now, David, is probably a really good report by exception, real-time data, and single source of truth.
And I think the other folks will get into that. Itâs nuanced, right? Where the confusion is, itâs part of the magic of it. And then the problem with it. But looking forward to the discussion.
David: Yeah, absolutely. Thank you. Welcome, Aron. Jeremy?
Jeremy: Yeah, Iâm Jeremy Theocharis, cofounder of United Manufacturing Hub. Because most of our customers are large enterprises, when we go in with United Manufacturing Hub, we need to convince hundreds of people about the Unified Namespace. So, what we like to do is make things very, very boring.
By boring, I mean Unified Namespace is actually nothing really exciting. Itâs just IT best practices applied. So what is it for us? Itâs an event-driven architecture with message brokers. Very simple. Very well known. So what is it not? Itâs, therefore, not a database because itâs not real-time. Itâs also not point-to-point.
But people typically understand with Unified Namespace that, at the core, itâs actually something very boring. A lot of people and industries already do it, but they just donât call it Unified Namespace.
David: Sure, absolutely. And Kudzai.
Kudzai: Cool. Thank you, David, and thanks, everyone. So yeah, my name is Kudzai Maditereza, Iâm a developer advocate for HiveMQ. I do a lot of evangelizing for the Unified Namespace. Iâm also a founder of Industry40tv, a media education company running a popular YouTube channel and podcast.
Unified Namespace is quite a nuanced topic but fundamentally is about the event-driven architecture behind OT/IT data integration. But a lot of times, I find that explaining it in mostly technical terms loses some of the audience.
So I think itâs more apt to talk about Unified Namespace, at least in my case, as being that data platform for continuous innovation. And I think weâll get a little deeper into that. For me, itâs the platform that gives real-time access to all data from all parts of your enterprise OT and IT, allowing you to continuously innovate on top of that data without having gone through a lot of friction around it.
So yeah, excited to be here and looking forward to exploring it some more.
David: Excellent. Well, itâs great to have you here as well.
And so I defined it as an event-driven architecture. There is a broker, and there are some DataOps in there. So Kudzai, could you lead us off by talking a little bit about the brokerâs role in this whole process? Why is Pub/Sub so important to a Unified Namespace?
Kudzai: Okay, to set the context, itâs important to talk about how we arrived at this idea of a Unified Namespace. Right? And what are the challenges, really, that weâre trying to solve there? I think that will help us paint a good picture of the idea of Pub/Sub and the broker.
Typically, a company has this idea of becoming a data-driven company because that helps address many different data use cases that you might want to address to maintain a competitive advantage. So whether itâs existing energy efficiency, OEE, or advanced analytics use cases you want to address, a Unified Namespace is something that, for you to address all these use cases, you need to connect it directly to all the different systems on the shop floor and to IT. So, youâve got this OT/IT integration whereby youâve got apps that are connected directly to equipment. So, each time you need to address a certain use case, you need to set up that connectivity infrastructure. Thereâs a cost related to that. And thereâs repetition also involved there. The time it takes for you to bring up, especially from different sources, up to IT, it takes a lot of time, months and even years.
The idea of the Unified Namespace is to say youâve created that hub of information where you have all these connections already set up, and youâve got all this information already being pushed to that central repository of information that gives you that real-time snapshot of all the, the current events and state of your business without having to worry about the underlying, connections. So, this idea that you are pushing data from the edge into this central repository of information and also from IT. Itâs information where you have this common format data unified and that is accessible through that single interface.
So, this is where the broker comes in. It acts as the platform that allows components or systems to connect from the edge and push the hub of information that then gives access to real-time events. And what really makes MQTT or the broker ideal for this scenario is that you donât want to just push data using it as if you are using a legacy protocol, where you just like saying the topic is say, a tech name, right? You want to be able to organize the data in a way that can be discovered semantically by anyone who understands the organization and the structure of the organization.
So MQTT gives you the ability to be able to organize information using its MQTT topic namespace. This is the part that the MQTT broker plays in the Unified Namespace. Itâs the source of real-time access to the status through a single interface without you having to connect to a thousand different servers on the shop floor. By connecting to one broker endpoint, you get access to all the unified, normalized data in a way that has already been organized and is being pushed using published communication.
I donât want to get in much depth about what Pub/Sub is, but at a high level, itâs the idea of the brokerâs role in a Unified Namespace architecture. So you still, at the end of the day, need some components that help get it out. And I think this is where Aron will be able to expand a bit on how you get that data out and then how you marry OT data to IT data within an MQTT information hub.
David: Before we move on to Aron, I want to make sure that I understand the brokerâs role. Itâs around this event-driven piece. But I need an easy way for edge devices or other systems that are going to instantiate that connection. So thatâs one of the first pieces of it is edge data can connect to something rather than having maybe a server client as the technology that I have to go and find all these. It enables easy access for all that data to get there. Then of course, now itâs that whenever thereâs new information thatâs part of the event-driven architecture as new information is available, I donât have to ask you for it. The Pub/Sub enables that new informationâyou just send it to me when you have it, and Iâll be sure that it gets out there. Is that a pretty good synopsis of the role of the broker here?
Kudzai: Yes, it is. And also maybe just to expand on that a bit, this idea of event-driven. As you mentioned you donât get to a situation whereby applications are reaching down to this job flow to ask for this data. Instead, it is pushed from the edge into the data infrastructure. What that allows you to do is create this idea of a single source of truth because all of the data that youâre currently viewing is the data that is actually being pushed from the edge, from the source of the data.
You can easily build your business model around it because itâs not data that has been aggregated through different layers. Itâs coming straight from the device at the edge into the data infrastructure. And, also the idea that you donât have an application that is pulling to get that information, it means that whenever thereâs a network failure with Pub/Sub, the edge component or the device is able to buffer that information and retransmit whenever that connection is restored, which in any other scenario, youâd lose that information whenever thereâs a break in connection.
So this is the importance of having that broker or Pub/Sub architecture in a Unified Namespace.
David: All right. Excellent. Yeah, I certainly donât want to overlook the importance of the ability to buffer data to ensure weâre getting everything there. So, I guess in that case, make sure that when youâre choosing a broker, you choose something that has that ability and all the technology supported.
So, Kudzai, you mentioned that as you were talking. Oh, go ahead, Jeremy, you had something.
Jeremy: This is more of a scientific explanation. We tend to go a little bit more theoretical. One of our most popular blog articles, we actually compared different MQTT brokers through it. We only got two cease and desist letters for that article, but itâs still online in its original form.
In the introduction, we try to give a more scientific reason why you need the message broker in industrial IT systems or, in general, in a system where you process a lot of data. You always have multiple building blocks. You cannot do everything in one application. And in order to connect those building blocks, thinking top-down, there are only three options on how you can do it.
You can send it through databases. Thatâs the first thing. So we have all the PLCs. They all send data in the database. Then you could do some batch processing on it. But as soon as it shows us that architecture, with a standard storing approach, you cannot do anything with real-time data anymore. All this stream processing. So data comes in and you change the color of a traffic light. Itâs not possible anymore.
The other way you can connect those building blocks is in service calls. So point-to-point. So you do a lot of point-to-point connections, and itâs actually a fairly reasonable approach. However, in manufacturing, a lot of these blocks tend to stay there for like ten years. At Google or Netflix, they can do it with point-to-point connections, but they also rewrite their whole applications every three years. I mean with factory stuff tends to be like this, then you have these spaghetti diagrams because you would never refactor.
And then thereâs only a third option, which is called asynchronous message passing. Itâs basically a message broker, which, unfortunately, adds a new component that can fail, which is bad. Then, the other side allows everyone to start talking to the message broker to exchange information in real-time. And then you put it in the database. So everything has advantages and disadvantages, but then manufacturing where the other two choices are mostly not a good way to go, a message broker is the only viable option left. Itâs usually a very good option.
David: It always helps when other people provide their own input. Certainly, a picture speaks more than a thousand words. It really helps me understand and unpack this, which is my favorite part of doing podcasts. I learn so much when Iâm on it. So, again, I canât thank everybody for their time and participation.
As Kudzai was talking, one of the things that came up was that we want to have these semantic data models. We need to publish them to a defined topic name structure. Because if Iâm just a device, and we all just start willy nilly doing things, I donât know that weâll ever be able to leverage the value of this Unified Namespace.
Aron, I think itâs HighByte that might have even pioneered the term DataOps. So, when I talk to people about HighByte and what it does, I say itâs a DataOps tool. I even mentioned that in my definition. So can you share with us what exactly is a DataOps tool? What is HighByte doing and why is it so critical to the architecture and design of a Unified Namespace?
DATAOPS AND THE UNS
Aron: Yeah. Sure thing, David. I think we started back in 2018. I think we looked back and did a Google search on industrial DataOps back then, and there were 70 results. And I think now thereâs 300,000. So we were talking about it pretty early. And obviously, thatâs changed over time. But still, itâs a pretty consistent message.
When we think about DataOps in industry, the classic example of a use case I like to give is a large industrial enterprise. I might have 50 facilities. I should be able to have a team of half a dozen folks that have the people that process the technology to manage my data movement from the edge to the cloud, in the cloud, back to the factory at scaleâwith the goal of driving the cost of change down to near zero.
So what does that mean? Hereâs a classic example. Some of our larger customers have AGVs in their factoriesâautonomous guided vehicles. So, forklifts drive around with nobody controlling them. A common problem with those is theyâll get stuck somewhere in the factory, and you need to be notified, and someone needs to go get them. Itâs kind of like your Roomba getting stuck under your kitchen chair, right? So thatâs a use case. Itâs a very practical one.
Letâs say you have 20 facilities that have AGVs across three or four vendors. What you do with the DataOps platform in that team of six folks is you can go in and say, âWhatâs the data we need from these AGVs to be able to detect when they get stuck?â
With that, you create a data model. And that data model is nothing more than maybe its location, last time it chirped data, maybe it was talking MQTT and a few other additional bits of information that say, âThis is the data we need from the AGV, regardless of the vendor, to be able to enable this use case.â
So, with DataOps, you can quickly enable that in one factory. Weâre not talking about months of integration. Weâre talking about weeks or less. You can then pipe that data to some system in the cloud. Maybe itâs going to Snowflake S3, maybe itâs going through MQTT as part of the UNS. Then, what you would do is youâd prove out that use case really quickly with the factory to say, okay, can we detect these AGVs getting stuck?
Once youâve done that, you should be able to take that use case, that data model and the technology you use to push that data in, replicate it across all of your factoriesâall 20âin less than a month, and enable that use case.
At the same time, you might find that you tried that use case, but it delivered no value. You just throw it away. So itâs DataOps of the concept of Iâm going to create this process, people and technology that allows me to test these use cases really quick and then either scale them up quickly or reduce them. This is part of the UNS story, too. Once you make this data readily available, people will start to use it, and then theyâll want changes to that data.
Theyâll say, âHey, we donât need these attributes in the data payload, but could we have these, or could we scan at a faster level?â The key to DataOps is ensuring people, process and the technology can make those changes at scale, at near zero cost. You should be able to add an additional attribute of data, wire that up across all your facilities, and publish that to your end system with very little work.
Thatâs the core of DataOps. We think about DataOps in terms of UNS, the part that applies directly to UNS, is that data model, that semantic model, like you said. If I have every PLC just chirping raw data into the UNS, itâs not very usable. Traditionally, factories donât have this concept of a nice rich namespace with a logical way we think of the factory. And then that data mapped to the actual data in the PLCs. MQTT and DataOps is a way to model that data at the edge to create those models and then move it semantically into MQTT. In this example, in terms of UNS, in a way that a human could come in and look at and say, âThis makes sense. I can see this site area line, this is the machine, this is the temperature sensor, regardless of the vendor.â
It abstracts away the details and the mess of the different implementations and technologies at the edge, so by the time it gets into the broker, it makes logical sense.
David: Excellent. So if I understand, we now have this broker. Weâre going to use MQTT in our example here for this Unified Namespace because it is fairly common. I have client instantiated data. I can update data as thereâs new information available. And I can also buffer that if for some reason thereâs a loss of connectivity thatâs there.
The DataOps piece comes into whatâs sometimes called data governance. Thereâs going to be a data contract between the producers and the consumers of the data. The âcontractâ will say Iâm not just going to willy-nilly publish raw data, Iâm going to create these agreed-upon, semantic data models. I always use a pump or a compressor, some type of asset to say this is what that payload is going to look like. And then Iâm also going to publish it to a topic structure that has already been defined.
So, commonly, weâll use the master data model. Thereâs a lot of references to ISA 95, using that hierarchy of enterprise site, area, line, cell. So we want to publish data thatâs there. Thatâs the piece that DataOps takes care of. We now have an enabling technology, but letâs make sure that weâre sending data in a way that everybody can understand. And if I needed to go find data, I would know where to get it. Does that make sense?
Aron: Yeah, exactly. And I think DataOps and UNS overlap quite a bit. DataOps is the flexibility to model and move that data wherever you want. MQTT happens to be a really easy, useful way inside the factory and even out for multiple subscribers to come in and access that data. So, it kind of gives you that semantic ability but also the flexibility to move it wherever it needs to go.
David: So, continuing on with many of the conversations around the UNS, one of the first questions that almost everybody asks early on is, well, is there a database? Where do I store my data? Because if itâs just fine publishing to it, does that mean that the UNS is a database? Actually, no, itâs just the current state of the business is the way I understand that. But maybe, Jeremy, you can help us understand a little bit about, does a UNS store data, or if it doesnât, then how do you go back in and retrieve some of that data?
Jeremy: Yeah. This is also something like our scientific approach to the message broker, and it always sounds a little bit unnecessary, but in situations like this, it really helps bring the discussion forward. So, I already said, there are only three ways we can connect building blocks. And if you do it by your database, then you donât have this real-time.
So, we already established the Unified Namespace in its core is not a database. It has a message broker. But then there comes the questions: With all the real-time data, I want to calculate OEE, I want to see the story. Where is the story in it? How would it work?
You have a Unified Namespace or a message broker. You have this DataOps or the contextualization layer, which helps you model everything, and then you have an additional database, so that whatever you have in the Unified Namespace, you can actually also put it in a database.
And there I think you can go deep down into rabbit holes, because what do you want to store? Because you can send a lot of information through the Unified Namespace. I would say the easiest thing is time series data. So you have it, then you send it through, and then you can define a certain schema if data is in a certain type of format. For example, if we have this topic, it will then light up in a time series database. And you can apply the same for other things. For example, if you have more of an ISA-95 style of informationâlike work orders, new product etc.âyou could store it in a more relational database.
Some people in the community try to combine it all. They say, âLetâs also put the database in there and letâs try to mix it up and put a graph database in there.â
From our experience, it just makes communication very hard. So, thereâs a message broker for real-time data, a database, and a microservice component that could also be the DataOps layer or something else. That component subscribes to the message broker to certain types of topics and then stores the information in the database.
And this is how weâve seen most companies approach it. Of course the exact tools changeâtype of databases, message broker. But with this approach, we aim to give it to any IT person, so that they will look at it and say, âokay, yeah, this is boring.â
Then, there is this step between what if I want to store data, but not permanently, only buffer it for a short period of time. For example, during connection outages this gets a little bit tricky because you donât want to use a database because theyâre only done permanently, but also MQTT, Iâve not seen actually broken yet with like a proper queue system in there.
So we use, for example, Apache Kafka for this as an addition. You could also do it in your own microservice that buffers the data in between. But these are the main considerations. Now, we could go into all types of details. I could talk for hours about this.
David: Thank you for that. So if I understand, weâre just limiting the Unified Namespace, and how we are defining it. Thereâs no persistent data or data storage. Itâs purely just for the Pub/Sub functionality, using a DataOps tool to make sure that weâre bringing in semantic data models to a defined topic structure. And then, depending on how you want to query or retrieve that data, that becomes more of a function of what youâre trying to do. Did I understand that right?
Jeremy: I donât know about the last part. Can you rephrase that?
David: Yeah. So now the endpoint depends on how I want to consume that data or query that data. Thatâs not something Iâm necessarily going to retrieve from the Unified Namespace. Thereâs going to be a database or some other piece of technology. Thatâs where I go to retrieve the data. Itâs just that everything has traversed that, and thereâs an agreement on the data Iâm going to find. Is that right?
Jeremy: Yeah, exactly. And what makes it really difficult is to ensure a consistent data model across everything. Iâm still writing an article on this thatâs half-published. It takes a lot of time because now you have data models in the PLCs itself. Then you have a data model in the UNS, and then you have the data model in the database. And you want people to still be on that set of the PLC, to still understand whatever happens to the database. So this is something that is really challenging, to design your data model so it works across all three of those components.
David: Yeah, I think thatâs a very fair statement. And actually, one of the things that I wanted to talk to Aron about is the ideals as weâre building our data models. This is very common in programming. We have this concept of inheritance versus composition.
INHERITANCE VS. COMPETITION IN DATA MODELS
David: When Iâm thinking of creating these data models, I might be inheriting something where Iâm a member. So Iâll go back to my easy examples, like the pump. I might just have a basic pump model, but some of the assets in my equipment, some of the pumps, now have more information available on them.
Do I want them to be a child of a pump? Or should I just create two completely separate independent models of this type of data? Or do I start bringing other data models and nesting them within an overall model? Suddenly, it can be pretty cumbersome and complicated on top of the three use cases you already mentioned, Jeremy. So, from a DataOps standpoint, Aron, how do you generally approach the data modeling and all these various scenarios?
Aron: Yeah. I look at those two extremes. You could start really simply and just model the bare minimum of what you need. We see some customers do that. We think they probably have more success being use-case-driven. As I mentioned, the AGV case. Start there. Go. Learn. Adapt.
Youâll have other customers that will go to the other extreme. I would call that digital twin, right, where itâs almost academic. Theyâll spend six months trying to model the factory perfectly, using UML on a bunch of stuff, and then trying to break that back into semantic models. Theyâll end up with the perfect model. Theyâll go to implement it and find out itâs really difficult. And hey, the end systems canât use this anyway because it has too much hierarchy, or we messed up here.
The sweet spot is probably somewhere in the middle. In terms of inheritance versus composition, we have a whole modeling guidebook. I think we wrote it a year or two ago, and itâs actually really useful. So I encourage people to go look at that. Thereâs a lot of practical advice there in terms of modeling, which is new to the space.
But in terms of inheritance and composition, with inheritance, we tend to lean towards composition more. And this is kind of academic as well. But the thing with inheritance is that you kind of create a hierarchy as a result of inheritance.
As you mentioned, David, you have a base model of a pump, which you derive for the Mitsubishi pump, and then different pump types. For the audience, you think of inheritance; you think itâs a relationship. So I have a car thatâs my base model car, and then I have a Volvo and a Subaru. I start to add attributes off of those base models and build that up. You end up with this implicit hierarchy thatâs somewhat coupled. What you end up doing is, if you go with the inheritance route, later down the road, you have to go change some base models. And there might be implications in that across all of your model definitions that you didnât quite anticipate. That can make changes more difficult.
So like the classic example, if folks are familiar with OSIsoft, now owned by AVEVA PI System. They have a product called Asset Framework, which is over ten years old now which is kind of a pioneer in modeling historical data on a story. And if youâre familiar with their system, they use templates, which are models, and they use inheritance. And if youâve ever dealt with a really large PI AF system, youâll instantly be able to tell why inheritance gets tricky. That coupling starts to make things really difficult.
So, composition, on the flip side, is a little more flexible. It just says, hey, I have a model, and I want to inject in that model a set of attributes that are, letâs say asset metadata. So, the name of that manufacturer, the asset, when it was last serviced, the year it was installed, that kind of thing. You can place that in any of your model definitions that are an asset, change that set, and have a smaller change set across all of your model definitions. Because only the things that are composed of that will change. So it gives you some more flexibility.
Composition is probably the better way to go, but itâs a great question. I would also say donât go digital twin. Digital twins work really well in building automation because buildings donât change. Theyâre built once, and thereâs a code and everything set up.
When you look at discrete process or batch manufacturing process, maybe not so much. But with a lot of data, things are in constant flux. If you try to design your models around digital twin to perfectly replicate that physical asset digitally, youâre just going to do a lot of work. And you might only leverage 20% of the value of that work at the end of the day, given the data you use.
David: Excellent. Iâve always liked the idea of inheritance because, from a data governance standpoint, it means I make one change, and I know thatâs going to proliferate or federate across the entire system, versus the composition where I might have to go in and do some other activities.
But I hear what youâre saying. Thereâs a lot more flexibility. I tend to be more on the composition side myself. It just means part of our data governance is when there is an update, weâll call it a best practice that everybody involved in this DataOps tool is responsible for updating their assets accordingly. Is that a good approach to handling that?
Aron: Yeah. And another practical example. You have an injection molding machine that has two motors, and then you have one that has three. You could do that inheritance, but doing that model via composition is a little easier. The flip side of that is again, you might not want that level of detail. By the time you get up to your cloud system and the AI and ML, because if Iâm coding against the data youâre providing, and Iâve got to deal with this one has two motors and three motors and five motors, I might not care. That just creates complexity for me.
So you always have to think about that end-consumer in your use case of the data and provide just enough thatâs needed to simplify their lives. So, pros and cons.
CHOREOGRAPHY VS. ORCHESTRATION OF DATA
David: Yeah. Very good. Great discussion. Thank you, Aron. So, continuing on the same idea, thereâs another concept thatâs known as choreography versus orchestration.
Moving this data around, how do we want to ensure that the right data ends up at the right place? So, the concept here is choreography. If youâre watching a play or an actual dance group, the choreography is Iâm going to do something based on what somebody else is doing, but nobodyâs necessarily controlling me or telling me what to do. I have to be paying attention to where everybody else is so I know where Iâm supposed to be and what Iâm trying to do. So itâs very independent versus orchestration. Think when you go to the symphony or the orchestra, the conductor is always providing that guidance. And I know Iâm not going to play my instrument until the conductor has looked at me and said, okay, now itâs time for you to play. And thereâs a lot more control over that.
The benefit of choreography, of course, is that all my systems can act independently. Thereâs a lot less overhead thatâs associated with that. Versus orchestration where now you have a system that is telling everything what to do, and it has to manage everything. And thereâs a lot that goes into that. In this conversation about orchestration versus choreography, Jeremy, what are your thoughts on that? How do you approach it? What do you think works? What should you try to avoid?
Jeremy: Yeah. I think we can approach this topic from multiple perspectives. Let me start with one, and then you can ask a follow-up question.
The first point is that on the topic of orchestration versus choreography, you could also rephrase it to something like central governance versus edge-driven. I would even say that orchestration, the central governance, is actually a failure mode for industrial IoT for big projects because this is how IT tends to think. Typically, in IT, problems are a little bit different. You have one cloud and one server; everything is very close. So, what you can do is you can have one central Git repo, and one central database. And from this, everything else builds.
I would say itâs a failure mode in manufacturing because in manufacturing, you are working with distributed systems. So what does it mean? You cannot assume thereâs always going to be an internet connection. The connections between all the components are often very unstable. You donât want your production to stand still if the internet connection goes down. Also, from a security perspective, you have all these demilitarized zones, and every layer acts independently from other layers. Of course, if there are longer outages, letâs think about the PLC. If there are longer outages, no information comes in from the SCADA system or the MES system. So it would probably stop after an hour, but it could work independently for a certain time frame.
Manufacturing itself is very much edge-driven. So itâs about choreography. There are a lot of different perspectives. The result of edge architecture is that itâs getting quite complex. If youâre designing a system and you cannot apply these typical cloud IT principles, and you start doing it, you might not be prepared for the result. Because how does it look then if you start with choreography, how does it end up? You have a lot of edge devices everywhere. There could be multiple message brokers. People in IT often ask, âWhy would you have multiple message brokers? Just have one.â No, we need every component to act independently from the others, so we will have a message broker at least four per side, if not even per production line.
Then you need to sync between it. And now, you can encounter a lot of different problems that arise from this architecture. I mean, the first thing is you could argue, okay, maybe letâs streamline everything. Maybe you can comment on that. I think itâs impossible to streamline that in manufacturing because everything is built to be edge-driven. So, if you want to work with it, youâre going into distributed systems. And there are a lot of challenges that arise that a lot of people also ignore. But as soon as they start to scale out, they realize this.
One example of data flow is what type of data youâre sending through the system. So now imagine we have a message broker for each production line and one for the site, and we have one in the cloud to gather all the data. If we know the multiple cases you have, for example, if you have time series data, itâs really important that 99.9% of the data arrives in the cloud.
If there is an internet outage, itâs important that the data is buffered so that it never gets lost and stored on disk. But to be honest, itâs acceptable if sometimes data points get lost. So if you send a data point every second, and then it gets lost once, okay. But to be honest, I think a lot of people have this problem, but they just simply donât notice because itâs fine.
But then, when youâre talking about things like adding a new order or sending commands, for example, in the cloud, you have a button to start a machine. Then things start to get complicated because how should the system behave if thereâs no internet connection? You donât want this command to get stuck somewhere and then execute hours later, so this is what makes choreography so difficult.
We try to build a product that gives you the overview over that. So you have all of these instances, you have all of these edge devices. We give you a single view that allows you to see the health of each of your nodes, the health of each of your network devices. So in case something goes wrong, you are notified about it. And you donât have to go through hundreds of VPN tunnels to get to that edge device; you just need to check that the problem is somewhere else. But, now Iâm stopping my monologue and getting over to you, David, for any follow-up questions. I hope this was what you asked with choreography versus orchestration.
David: Thank you for that. I tried to pick words that were going to be very difficult for you to say. And Iâm glad I was successful with that.
So, your comment about orchestration versus choreography is that you were talking about the pitfalls, especially if you do a full-blown orchestration where the command is central to that governance.
I wondered if maybe we could come back to the Goldilocks principle. Can there be a balance? There will be some choreographed systems, but other systems are more orchestrated because we want to have more control over that data and understand whatâs going on. Would that be a good approach? Maybe we could try to keep it as choreographed as possible but add orchestration as needed.
Jeremy: Yeah. So what we try is to bring both worlds together because, at the core, it needs to be choreography. Everything needs to work independently of each other. But on the other side, just from a maintenance perspective, you donât want to have all this chaos. You want to have central view. You want something like Git and GitOps to configure it. So you need to meet in the middle. Our current hypothesis is the data infrastructure needs to be distributed. But then the configuration of that can still be centralized. So that would be the current hypothesis on where the trade off needs to be.
David: All right. Excellent. Thank you for the follow up on that.
So, another question that we get asked a lot, and I think Kudzai, you can speak to this, is about some of this conversation around inheritance and composition or orchestration versus choreography.
QUALITY OF SERVICE WITHIN MQTT
David: The conversation is all about needing to know that my message made it to where it was intended to be sent. While weâre not here to design a point-to-point per se, I want to know that if I have a change that is relevant to some other system, there needs to be some assurance that that message got there. What comes up oftentimes is, specifically within MQTT, this concept of your quality of service or your QOS.
So, while QOS2 will ensure that the message makes it there, my understanding is that it is really just the relationship between the client and the server, not necessarily from the client to the other client. So, can you just talk a little bit about the quality of service in general and its implications? And then, for those that must make it there overnight messages, what does that look like using QOS2 between two clients?
Kudzai: The same applies. Thatâs the same question that we often get from customersâthey want to send MQTT commands. And just before I go into the QOS, most of the time, we basically tell the customers that if you need to control your robot, like you need to perform some kind of closed-loop process control, you are better off using OPCUA or some other field-level protocol because it guarantees synchronized communication, where you get the response, the immediate feedback about whatâs going on. So, MQTT is really not built for sending those critical commands or messages.
As far as quality of service is concerned, yes, you are 100% right. The relationship between the quality of services between the client and the broker. With quality of service, the broker guarantees message delivery to the actual client. What that means is that when the publisher sends a message, it will get an acknowledgment that itâs been received by the broker and that the message is being sent to the client. However, what happens after the message is handed over to the client is not something that you get within the MQTT framework. So this is something that would typically be addressed in a direct, point-to-point protocol. However, with them out of service, there is the guarantee that even when the network fails during that exchange, all of the clients that did not receive that message during that network disconnection, the broker would follow up with all those clients and make sure that they do send an acknowledgment. Then that actual acknowledgment is sent back to the publisher or whoever is interested in knowing what acknowledgment has been sent. So, that information is not lost. Right?
So the quality of service guarantees you all of that information as long as youâre using persistent sessions. But again, if you want to control a robot or if you want to get data out of a database and search through what happened, you are better off connecting directly to that application and then that snapshot of information, that view, you could then publish it to the MQTT, Unified Namespace for all the other components that might be interested in that result. So, MQTT is only for real-time events instead of across your entire enterprise and not for transactional interactions.
Jeremy: To follow up on what Kudzai said, first of all, please believe me. I studied mechanical engineering, so I studied the vibrations caused by CNC machine milling. I realized that sometimes referring to scientific IT literature. And taking a look at, hey, how do these big companies solve it? That really helps.
I think this question of quality of service is interesting because it is already kind of like a soft problem. So what âquality of serviceâ means is that it guarantees the communication between the client and the broker that it happens only once. But still, it doesnât guarantee that through the system. For example, if you have one message broker that originates from one client, it actually ends up at the second client. Or even if you have multiple message brokers that actually end up through multiple brokers as a bridge to the cloud, whatever, it doesnât guarantee you that. So, you always need to solve this on the application layer.
Itâs not the problem of MQTT. Itâs just how IT protocols work. Even the underlying fundamental part, even in TCP, you have this concept of resending messages, but it only goes from the network stack of one device to the other network stack. So if anything gets lost there, theoretically, thereâs already a retry mechanism in there. Also with MQTT, every day, there is a retry mechanism, but still in MQTT thereâs the option that the message gets delivered somewhere. For example, it gets delivered to the client, but something goes wrong between the delivery of the message and the actual processing of it. How we typically solve that, and how we can send commands through the system is by giving the message a certain type of payload, to actually design the protocol and the application level to actually ensure that some of the problems donât arrive.
For example, you would have, like, a request and response mechanism even through MQTT. And if you really need to ensure that something was really executed, maybe we can link to the article further down. It really goes well into the details of exactly this quality of service discussion. So if anyone is having exactly these type of thoughts, I can definitely recommend going into this article here.
David: Yeah. You shared that article during our conversation about this, and I found it exceptionally helpful because it brings everything that weâre doing full circle. One of the questions and the struggles that I have had is that weâve always talked about this Unified Namespace both for the data and the events and transactions. And to me, these events and transactions are asynchronous. And itâs not like, say, time series data thatâs just constantly flowing in. As we talk about events, itâs either the car arrived somewhere, thatâs an event, or you could put an event frame aroundâwe had a production time or downtime or thereâs a shift. An event that could be time-bound is another aspect of it. And then, of course, there are transactions that are at the end or the beginning of those events. There are certain payloads that publish things off.
So, you know, maybe these are just last questions. In general, is a UNS really geared for time series data, or can it be used for events and transactions? If so, what is the best way to approach the events and transactions? So at this point, anybody can chime in on thoughts around that idea.
Jeremy: I think that the UNS is suited to some type of degree for events. Iâm still investigating this, but my gut feeling tells me the more complicated it is, the more itâs better to have it in a different type of system.
One example, a state machine, you know what to enforce, that the production is always in a reliable state. So that means a line PLC or a SCADA system. So if you want to use UNS as a communication through everything. So, send a machine command, and you can show with the techniques that I mentioned that data actually arrives there.
But who is taking care of the state of the other production line? So, what do I mean with state? When the machine is starting, in order to go from the âstartingâ state to the âstartedâ state, PLC must first respond with a certain type of command. There must be a reset. Whatever. This type of logic of events. Itâs much better suited in an external system because UNS cannot handle the state thing, but you can use the UNS to send this information through it.
I donât know if it makes sense. And maybe others can chime in when you have some more experience in this regard.
David: Yeah. I think itâs going to be a topic that certainly gets discussed on future podcasts. So, we are a little long on the podcast. I do apologize. I believe Kudzai, you need to head out real fast. Do you have any final closing thoughts? We started off with âletâs define a UNS.â Do you think weâve achieved what it is we set out to do here?
Kudzai: Thank you. Yeah, I think we did. So Iâll quickly go through some final thoughts about the Unified Namespace and the idea of events and transactions. Again, just to reiterate, MQTT is a protocol that gives access to the real-time events and state of your enterprise. Itâs not so much used for controlling robots or accessing the database. That needs to be handled outside of the Unified Namespace with a direct connection to the database or a robotic system using a field pass protocol or Rest API endpoint, whatever the case may be. So MQTT holds that current state of information, and using features such as retained message, it makes sure that youâve got Unified Namespace always living within your MQTT broker, not as a database, but just as that snapshot. So, whoever joins the network is able to discover what the current state of any component that is connected to the Unified Namespace is.
And if the network happens to disconnect on any one of the clients, that information persists, but not as a database. Again, just as a mechanism to make sure that that information is redistributed whenever these different components rejoin the network.
We spoke about the database connectivity aspect earlier. For example, for HiveMQ, we provide a connector that allows you to persist all of this Unified Namespace. So you could say, select different namespaces you want to consume where they connect, Iâm going to say Snowflake. If you want to sit in a namespace that you want to persist to, you use a connector that allows you to do that. If you want to persist data to a MongoDB, thereâs a connector that allows you to do that. Or if you just want to send it over to Kafka, thereâs a connector that allows you to do that. So MQTT itself doesnât persist, but it gives you a mechanism to be able to persist your data outside of the broker.
Lastly on the idea of orchestration and choreography. So I mostly like to think of it as it helps to look at Unified Namespace as a way of sharing data rather than collecting data. I think weâve been accustomed to the idea that data is a byproduct that merely needs to be collected. But itâs the mindset shift where you start to look at it as this idea of having to share data, which really encourages owners of that data to actually package it in a way that is usable on the other end because now it is being shared rather than just being made available for collection.
And itâs kind of like a mix compromise. We have central governance, and youâve got this distributed approach to it. So itâs a balance that needs to be struck.
David: Yeah. So it sounds like thereâs a lot that goes into a Unified Namespace. And the ways it can be approached and depending on the use case really will indicate how you go about it and actually developing the formal architecture to it.
So, Kudzai, if you need to drop, please do. I do appreciate your time. Thank you for that. But before we sign off, Aron, weâre talking a little bit about time series data, versus events and transactions. Any thoughts that youâd like to offer on the topic?
Aron: Yeah, as you say, hats off to Kudzai and Jeremy for their time. I think to say MQTT, donât try to do transactions through there. Donât try to do commands. I think it is a message we kind of need to deliver to the broader market. And itâs refreshing to see that because we are in the early days of UNS. Itâs even on the Discord channels and stuff nowâpeople trying to figure it out and itâs like, just donât do it. In most cases, itâs not the way to do those.
Whatâs interesting about the point-to-point connections is that you need a guarantee the data is going to be delivered to S3 or some other source, and youâre running it through MQTT, can you kind of hack it, using the client IDs and stuff to say, âhey, if this client drops, then start storing.â Yeah, you can do it. But it might be a sign you might want to send that data to an alternate path. Maybe that should go to MQTT, but it should have a direct connection to S3 or Blob to guarantee that delivery.
So what I think is interesting is UNS has caught fire. Itâs definitely here to stay. I think MQTT is a core component of that report by exception. But you know, all these conversations and confusion is really the market saying this is great, but what about this? What about this? And what about this? Because we have slightly different use cases in manufacturing, MQTT design for oil and gas, and the protocol itself doesnât encompass all of those.
And Kudzai was very honest about that. And I think thatâs great. I think MQTT is a core part of UNS. And I think the interesting part is how does UNS grow? How do us, as vendors and technologists, support our customers to enable these other use cases through best practices? I think, David, itâd be interesting to come back in six months and be like, okay, what did we say that was right? And what do we say thatâs now changed? I think it is moving that fast.
David: Oh, thatâs an interesting perspective. I hadnât thought about how we could revisit this. And what I said at the beginning of this call is, well, actually thatâs not what a UNS is. So, I think maybe the struggle is that weâre still aligning ourselves on what the technology is and whatâs in the approach because the intent is that we want to create these data-driven organizations. We want them to have access to all of that data in real-time, and they can use that for solving problems, whether itâs asset health or predicting the failure of a piece of equipment. And what does all that look like? So, before we sign off, Jeremy, any final thoughts you want to share with us?
Jeremy: Yeah, to add to that, the opinion I stated is clear. So, I said that it would be fine to send some comments to the UNS. But for more complex things, itâs probably not that well suited. But for controlling the production, you can still use UNS, but itâs not that intuitive to do. Just sending messages through it, thereâs a lot of complexity behind it. So if youâre unsure about whether to do this or not, I would tend to say probably for now, you can use it, but be careful of what youâre doing.
David: Excellent. Thank you. And really, Kudzai was talking of Industry40tv or the HighByte website, and certainly, with UMH and Jeremy, thereâs a lot of great content about things to think about as far as weâre doing this.
So, at the beginning, I define the UNS as itâs an approach to an event-driven architecture that utilizes a Pub/Sub technology like MQTT, along with a DataOps platform, that weâre creating semantic data models and publishing it to a defined topic structure. It seems like thereâs a lot of other considerations in terms of composition versus inheritance or orchestration versus choreography and taking advantage of some tools that are in there. Just make sure that what tool youâre using is the right tool for the job.
Well, thank you, everybody, for your time. I want to thank the other people who were involved in all these conversations as well because itâs certainly something that Iâm trying to get my head wrapped around. But thank you, everybody, for your time. I look forward to seeing you on our next podcast.
