Resonai Team Spotlight: Zachi Karni, Software Developer

Augmented reality holds immense potential for enterprises, but extracting its full value requires a unique skill set. You need to scan each room from multiple angles, translate the relevant data using an algorithm, and generate detailed 3D models that any end-user can navigate. In practice, this is only possible thanks to people like Zachi Karni, a software developer at Resonai, whose efforts let us render entire buildings at scale and create living digital twins.

We recently spoke with Zachi to learn his perspective on Resonai’s rendering process and how the technology is transforming the built world.

Interested in joining the Resonai team? Check out our open positions and apply today.

What We Learned:

• At its core, AR is a digital reconstruction of human vision. The most effective way to render any computer model is to assemble visuals much the way the human eye does.
• Scale is the biggest AR challenge. Rendering techniques are relatively standard across industries. The true challenge is recreating massive building complexes quickly, accurately, and at scale.
• “Everything that can become digital, will become digital.” This quote from Indigo founder Benny Landa was about publishing, but it also applies to facility management.

Hi Zachi! I’d love to open by learning about you and your role at Resonai.

My name is Zachi Karni. I have a Ph.D. in computer science in the fields of computer graphics and vision. In Resonai, I’m primarily in the R&D team, working on a rendering process that takes a complex model of buildings and generates images from it. And this rendering is done in the cloud at a vast scale — we’re taking a massive structure with a huge model, and we want lots of snapshots to build the reference for our algorithm.

How does this process work?

So the complete process consists of multiple stages. The first is scanning — in order to generate the building model for the digital twin, we scan the building using several devices and techniques. One example is using a 360-degree camera to capture everything around that camera position. This step produces a considerable volume of images we can put into our algorithms to produce data for our model, addressing any semantic details.

At the rendering stage — where I do most of my work — we want to simulate how the human eye creates an image from these data points. So let’s say you have a picture of a wall, some tables, and some objects. It may also have light coming into the room, bouncing from the different surfaces.

The basics of rendering are about simulating this physical process to create a unified image, but there are levels of accuracy. If you want to render the image quickly, you can make assumptions that aren’t based on reality, such as where the light is coming from. That’s okay; It’ll still get a very nice result. Maybe I want to go a little further and include a better simulation of lighting and other physical phenomena to get a more accurate result. But this requires different rendering algorithms. For example, it can use simple projection-based approaches. Or it can move up to ray tracing or photon mapping to simulate physical methods of preserving the light.

Once we’re finished, I don’t have to go to the building anymore. Instead, I can view and walk through it as a synthesis of digitally-recreated images from my iPhone.

Okay, that’s really fascinating. What about this process is unique to Resonai? Do you use a special rendering technique?

The first difference is the scale. You need a considerable amount of data to convert a modern building complex, which requires many images from different positions. We also need it finished quickly, and it must be accurate. The most challenging part is that it’s not one thing — it’s everything at once with scale as a common factor.

The second reason Resonai stands out is this rendering doesn’t require additional hardware, even at this large scale. In a video game, you can process all this in local memory. However, once you add a large-scale model, you cannot do it locally, so we deploy rendering applications across the cloud. So, for example, you can have a single thread rendering from one location, another in Tennessee, and all parts collapse to generate the final image of the building.

We also want to divide rendering tasks in a smart way so the entire process is efficient. One of the things I do is split the model into several blocks and render each block separately. It requires looking at the whole space and asking, “Okay, what’s contributing to my image?” Then I can process in the cloud, take all the contributions, and glue them together into the final image.

What excites you about augmented reality, computer vision, and the work you’re doing at Resonai?

In this business, and computer graphics overall, it's a lot of programming and sorting through code, but in our work you see something at the end. You actually see what you've been doing. You see the result of an image, while in other fields of computer science — which are very interesting as well — you are programming and will see some numbers and data.

The first time I was introduced to computer graphics, I was studying mechanical engineering. We took a course in computer-aided design, and I was so amazed by the vision. I said, "Okay, this is what I want to do long-term. This is what I want to do for the rest of my career." I didn't switch; I entered another bachelor's degree in computer science just to go into this, the process of learning how to program efficiently. Of course, there's a master's and Ph.D. and everything else. But for me, it was the end result that you could actually see what you're making.

What kind of future do you see for augmented reality?

There is a smart sentence that Benny Landa, the founder of Indigo, said a few years ago. He was talking about the printing industry, which was old-fashioned — you had the old printers and a huge machine that outputs a lot of volume. And he had a vision that it needed to be digital. And everyone said, "Come on, you are crazy, this is not needed. What we have is good." But he pushed ahead with the transformation.

What he said was, "Everything that can become digital will become digital, and printing is no exception." And I think Resonai is in the same game, just not for printing. Building management and maintenance are going into the digital world. And I think the Resonai platform is what's going to allow this.

Thanks so much to Zachi for talking to us about his experience! Are you interested in joining the Resonai team? Check out our open positions and apply today.