In 5 to 10 Years, Using Weapons Without AI Will Be Considered Unethical | Yaroslav Azhnyuk, The Fourth Law

[00:00:00] [SPEAKER_01] There's lots of conversation about AI and defense. What we should be really talking about, I think, is software and defense. We started with making hammers for pets and now we're making hammers for drones and we're throwing explosives to people who came to occupy our land. I think like within 5 to 10 years it will become unethical to use weapons without AI. The same way as it might become unethical to use cars without AI. Because those that are less lethal and non-AI weapons will be more likely to cause collateral damage, not less like them.

[00:00:29] [SPEAKER_01] But my only question to these companies is, if you're so great and powerful, why aren't your asses helping us fight out the rushes? Why aren't they on the front lines?

[00:00:41] [SPEAKER_04] Okay. So then you decided to get involved in defense tech.

[00:00:49] [SPEAKER_01] Actually started a second company in defense as well, which makes cameras. So it makes thermal cameras, which are now some of the most popular thermal cameras in the military. They're used in interceptors. It was probably over a couple thousand heads down with drones with these cameras. And they're used in just frontline FPEs as well. Yeah. So it's funny that there are some people from Tycube who also helped to start this company.

[00:01:17] [SPEAKER_01] And because they were doing cameras for the last 14 years, cameras for pets.

[00:01:25] [SPEAKER_04] Oh, that's right. Yeah.

[00:01:28] [SPEAKER_01] They have a lot of experience with that. Right. So it's so ironic. Like, yeah. Like we started with making cameras for pets and now we're making cameras for drones. And we were throwing treats to dogs and now we're throwing explosives to people who came to occupy our land. So that's Faith Love's irony for sure.

[00:01:53] [SPEAKER_04] There seems to be a focus here. You're doing counter drone technology.

[00:01:59] [SPEAKER_01] Yeah. I mean, with the fourth law, we're working on massively scalable systems for drones. And at this point it involves the like frontline FPV drones and then the interceptor drones and some other drones as well.

[00:02:20] [SPEAKER_04] The interceptor drones, are those for FPV or are those for Shahid or what?

[00:02:27] [SPEAKER_01] Yeah, that's a very good question. So, probably like the most dynamic area that's had interceptors. But obviously, when you're talking about interceptor drones as class, that can apply to intercepting tryheads, ISR drones or reconnaissance drones and then smaller drones like FPV drones. So frontline protection. So that's a very important question.

[00:02:57] [SPEAKER_01] And Eric Schmidt's also working on interceptors for Shahid.

[00:03:01] [SPEAKER_04] Is his, at Swift, is his company only looking at Shahid or are they doing the full range?

[00:03:09] [SPEAKER_01] Well, I think from the public sources that they've been telling that, I think there are like at least three initiatives. One is Shahid interception. Another is frontline autonomous drone. And the third one is something like a mid-strike drone. So this strike drone capable of maybe, I don't know what is theirs, but mid-strike as a category is up to 200 kilometers.

[00:03:39] [SPEAKER_04] Right.

[00:03:40] [SPEAKER_01] Yeah.

[00:03:40] [SPEAKER_04] And they're very capable, I think. Yeah, certainly. I mean, Eric Schmidt is no one more capable on AI.

[00:03:50] [SPEAKER_01] They're our main competitor slash peer. So like we don't strictly think of these companies as competitors because they're sort of trying to accomplish the same goal as we are. But yeah, obviously this episode in the Silicon Valley, the show where they're like, okay, these guys are making work better in place, but we don't want them to make this better than we're making work better in place. Something like that. It's not like that. So, but there is three some funding competition there.

[00:04:16] [SPEAKER_01] But I think that what they're doing is like, for example, far ahead from what any other company outside of Ukraine was able to demonstrate, far ahead of say, Andrail or Alsa. Right. And the... I think we're close to them in terms of the ground economy. It's like on ground, so sort of actually the economy. And we're somewhat behind in terms of the aerial interception. Yeah.

[00:04:45] [SPEAKER_01] I think obviously they have a minimal divergent side. Yeah. I think we have some of the best engineers in Silicon Valley, et cetera. And like, at the moment I try to do back-up and hundreds of millions of dollars.

[00:05:00] [SPEAKER_04] And he has this incubator with D3, right? Yeah, it's like full disclosure.

[00:05:06] [SPEAKER_01] I was somewhat involved in...

[00:05:08] [SPEAKER_04] In the D3 incubator.

[00:05:10] [SPEAKER_01] It's a small group of people who were sort of at the inception of that.

[00:05:15] [SPEAKER_04] Right, right. And is that funding primarily defense tech or are they funding everything?

[00:05:23] [SPEAKER_01] Yeah, I think their public stance is that they're funding defense tech. And it doesn't have to be necessarily a Ukrainian team. So I think part, like, about half of their portfolio is teams originally from Ukraine, and another half is not in Ukraine. Right. Because... It's relatively small by Eric Schmidt's time. So I think it's like a couple dozen million dollars. So it's like very small money.

[00:05:50] [SPEAKER_04] Is that how much?

[00:05:51] [SPEAKER_01] Maybe like, it started with like 10 million dollars. Oh, is that right? Maybe now tens of millions of dollars.

[00:05:59] [SPEAKER_04] Yeah. And do you have any sense of their production versus your production? I mean, how many interceptors they're... Nope.

[00:06:09] [SPEAKER_01] Well, like, this is not something that you're supposed to sort of disclose in public. We do have sense of course. You have what?

[00:06:19] [SPEAKER_04] I think people in the industry do have a sense. Have a sense. Yeah. I mean, but is he full production now? Are you full production? Are you still ramping up?

[00:06:28] [SPEAKER_01] Well, you're like, I think everyone is ramping up in this world. Yeah. It's like, obviously, they're called narrow, the interceptors by Eric's company. And I think they've started selling into Poland and Romania. I think they had some actually like production roadblocks for Ukraine. I'm not sure, you know, what the plan is with their activity in Ukraine.

[00:06:59] [SPEAKER_01] They're also very capable interceptors by other companies, which are not as advanced with AI, but those companies are intercepting and they're buying AI modules from us. Oh, so... That's a very different approach. I would say what Eric is doing is more kind to sort of Apple strategy. They have the full vertical integration. They ship that to you.

[00:07:26] [SPEAKER_01] And what we are doing along with some other Ukrainian makers is more kind to Android strategy where different teams work in different modules. So, for example, we provide autonomy modules which are growing in their capacity and those modules can be adopted onto various types of systems versus by the modern module from that extent.

[00:07:49] [SPEAKER_04] Because I was trying to understand where their models are coming from, where their chips are coming from. Yeah. He was saying they produce them in Ukraine. Is there an adequate semiconductor manufacturing or assembly in Ukraine to provide chips?

[00:08:06] [SPEAKER_03] Hmm.

[00:08:07] [SPEAKER_01] Yeah. I think he might have misunderstood the question. There is no semiconductor manufacturing in Ukraine. Maybe he was talking about the circuit boards. The assembly, yeah. Yeah. So, you can do sort of circumferent bird assemblies in Ukraine.

[00:08:27] Yeah.

[00:08:28] [SPEAKER_01] There are some facilities where you can do up to eight layers, but I don't think they are necessarily using them or... Well, it doesn't matter. So, you can do a lot in Ukraine, but then in terms of the actual chips or the sensors, like we're using sensors for thermal cameras, these currently have to be bought from... made in semiconductor fabs.

[00:08:57] [SPEAKER_01] There are some plans to do semiconductor manufacturing in Ukraine, including our plan of building a thermal sensor factory. But that is still planned.

[00:09:11] [SPEAKER_04] They have this rocket-shaped form factor that doesn't fly like a cruise. Are you guys using something similar or are you using the traditional...

[00:09:20] [SPEAKER_01] We're like, as a maker of autonomous systems, we're providing modules which can be integrated onto this form factor. So, the most successful Ukraine interceptor is a company called Wild Hornets. And they're producing this drone called Sting. So, it's a very similar form factor to General Cherry's. In fact, it was a similar to a scandal where Sting said that General Cherry just copied what they did.

[00:09:50] [SPEAKER_01] It's more or less like, you know, you made a car, so you copied my idea for a car. It's hard. It's an IP that's hard to protect. So, we're working with all of these guys. And then there are two major categories. One is sort of what we call a rocket drone or sort of a tail-sipper drone with these four sort of wing propellers. Another is a more classic sort of plane form factor.

[00:10:19] [SPEAKER_01] And there are a number of companies which are making planes.

[00:10:24] [SPEAKER_04] Fixed wing, you mean?

[00:10:25] [SPEAKER_01] Yeah, yeah, yeah. Sorry. Fixed wing, UEDs. So, we're working with both of those. When you say autonomous, how autonomous are these? Yeah, like there's a rabbit hole here. So, I sort of have this classification that I suggested and most people in the market sort of accepted this. Something that makes sense. So, when we talk about the individual autonomy for a single drone, we talk about these five levels of autonomy.

[00:10:55] [SPEAKER_01] And when we talk about the ecosystem, we talk about sort of eight dimensions of autonomous of the system. So, the five levels for autonomy, which are similar for five levels for safe self-driving. And you have level one is autonomous terminal guidance. So, whenever you see the target as a pilot, you're flying as a pilot, non-autonomously manually, you target log. And after that, the drone is capable to autonomously hit a target in all the possible circumstances.

[00:11:25] [SPEAKER_01] That's relevant all the time. Level two autonomy is sort of the same, but instead of hitting it kind of kind of style, you do bombing. So, you calculate one to drop the ammunition and you do the escape one. Right. So, that would be level two. Level three would be when your system can identify the target autonomously and make a decision to start.

[00:11:51] [SPEAKER_01] So, it's flying and then it scans whatever it sees through its camera. And then when it sees, it knows that it is now in a mode where it's sort of in a kill zone to find a target. When it sees a target, it first recognizes it and then gets enough confidence to make a decision to start strike, which would be, again, either level one or level two or kind of cuts out on that.

[00:12:15] [SPEAKER_01] Level four is navigation so that you can navigate all the paths from the unit where you start to work. The residual zone where you need to find a better gun. Level five is autonomous takeoff in length. So, it could be bombing and retrain the land as well. So, that's for sort of for flying drones. You can sort of adopt it to any kind of autonomous vehicle.

[00:12:38] [SPEAKER_01] And then when you look sort of further about the, like for us, individual autonomy is imperative. It's very important to accomplish and it's very important to accomplish in mass. So, if the system that allows the visual autonomy costs even a couple thousand dollars, it will be difficult to use such system on F3 drones which cost just $100 per piece.

[00:13:06] [SPEAKER_01] The system has to be relatively inexpensive and also relatively easy to manufacture so that it can be manufactured in millions. Because that's what the money warfare demands. And then if you look further, the individual autonomy, you have sort of this autonomous battlefield. There you have these eight dimensions of autonomy as we say. So, dimension one is levels of autonomy, five levels of different structures.

[00:13:35] [SPEAKER_01] Dimension two is the kind of carrier that you use. So, it can be a clock doctor, it can be an FDV, it can be a larger survivor. So, it can be a fixed frame. It can be a missile. It can be even a shell, a nuclear shell. It can be a man-sea vehicle or an undersea vehicle or a man-brand vehicle.

[00:14:02] [SPEAKER_01] The third layer, the third dimension is domain. So, if you do, you can use quadcopter, but it can be in ground-to-ground domain against enemy, infantry, or tank. It can be ground-to-air and interception. It can be ground-to-sea, which is quite a different task. And all the combinations of domains. So, that's sort of the third dimension.

[00:14:31] [SPEAKER_01] The fourth dimension is sort of what we call higher levels of autonomy. Things like swarms, bone carriers, bone nests, etc. Then you have, like number five would be different outside conditions. So, day, night, winter, summer, different types of targets.

[00:14:55] [SPEAKER_01] Because attacking an infantry unit or attacking a tank or a vehicle are slightly different tasks, which require some different developments. Then you have, level six, you have command and control systems. So, as a commander of a brigade, which is like 2 to 5,000 people or even ports, which can be up to 40,000 people.

[00:15:25] [SPEAKER_01] How do you manage all of your units with these autonomous assets? How do you manage all the data coming from those assets? How do you command, analyze, etc.? Then on level seven, you have all the infrastructure needed for all of this to work.

[00:15:43] [SPEAKER_01] Starting with simulations, which are used routinely for testing, etc., to things like data collection mechanisms, security subsystems, remote deployment subsystems. And then finally, sort of dimension number eight is how massively deployed that system is.

[00:16:07] [SPEAKER_01] Because if you have one system deployed in hundreds of thousands of units, another just has to field test provisions and are just being tested. You know, a couple hundred units being tested. Those are very different systems, very different levels of capability. So, that's sort of how we think about this whole ecosystem. I think it's like what's going on here is just astounding. This is like the Defense Valley. Right, exactly.

[00:16:35] [SPEAKER_01] Like here now in this room, there is a guy who like sits somewhere here, who works in the industry, whom I know very well. And this happens to me all the time. Like I come to meet some Defense Technology founder, and then there is another Defense Technology founder who works into it. And we introduce each other. So it's a very, very like Silicon Valley vibe.

[00:17:00] [SPEAKER_04] Yeah.

[00:17:01] [SPEAKER_01] You know, lots of confidence demands. The ideas flow. Yeah. Lots of expertise. So, even with the domain expertise. Now, I say this all the time that my definition of strategy is that strategy is a plan of victory using the uniqueness of your situation. The uniqueness of Ukraine's situation is that we've been at war with Russia for 12 years. And no one else in the world wants to steal that advantage from us.

[00:17:30] [SPEAKER_01] So, you have a couple million people who are one way or another involved in this. So either as fighters or as makers, engineers, entrepreneurs, or as assembly workers, or as volunteers, operation specialists, governmental clerks. So this is an incredible domain experience.

[00:17:56] [SPEAKER_01] And it will remain so for decades to come, even if tomorrow there is a ceasefire. And like, for me, it's obvious the world will be rearming for the next couple of decades, driven by supply chain instability, by obviously advances in drones and robotics, and obviously by AI. So, three huge factors that will drive the global rearming conventionally and unconventionally.

[00:18:25] [SPEAKER_01] I think we have probably another dozen or dozen and a half countries in the world who are now working on their nukes, who do not have an official nuclear status. So, unconventionally, I think that will also proliferate. Again, because of sort of Russian invasion in Ukraine and the Depeche memorandum and sort of the current policy, the foreign policy by the US.

[00:18:51] [SPEAKER_01] Now there is this very unique moment in time for Ukraine, which can be a cornerstone of not just Ukrainian but European strategy, building the Florence of the fence here. And this is happening. I think the government can definitely accelerate that if they pass some smart regulations. But one way or another, Ukraine and Ukrainians will play a critical role in the rearmament of the West.

[00:19:21] [SPEAKER_04] Do you think that the industry will turn back to commercial consumer tech after the war? Do you think that enough defense of a defense industry has been built that Ukraine will become a supplier?

[00:19:37] [SPEAKER_01] Absolutely. But I think that when I hear after the war, I imagine something quite different. If we have ceasefire with Russia in 2026, this is not after the war. Right. After the war is when the global arms race is good to extend. So before that, unfortunately, in like the next couple of decades, the whole world will be a rearming and now one piece prepared for the war.

[00:20:07] [SPEAKER_01] So that's what everyone is going to be doing because these systems, they just change the equation so much. Like you can take out a $5 million tank, you know, the drone costing $500. Yeah. And if you can add autonomy to that drone for the cost of a couple hundred dollars on top. And, you know, and autonomy itself makes every drone usable by a hundred times as many people. Right.

[00:20:32] [SPEAKER_01] So you don't need to train as pilots makes 10 times more precise, makes it not one time kind of passive with multiple time bombers. There'll be 10 times more utility. So you guys basically order over 10,000 times improvement by adding complete autonomy to the system. And $5 million dollars and $500. How many orders of magnitude is that? It's like a thousand, 10,000.

[00:20:59] [SPEAKER_01] So it's 10 to the fourth multiplied by 10 to the fourth is like 10 to the eighth. It's like what? 10 million, 10 million times, 100 million times more people than current systems. It's like, it's hard for human to imagine what that is.

[00:21:22] [SPEAKER_01] And then like, if we talk about threats, like in September, there were 20 Russian Shahad imitators. Herbera, what they call it, flying over Poland.

[00:21:45] [SPEAKER_01] Only four of those were shut down with equipment that cost probably two orders of magnitude more than the assets. General Secretary Rutte had a press approach saying, We did a fantastic job. We shown resolve of NATO, how we can protect our land. But it's obvious to everyone, it was a huge disaster. Yeah.

[00:22:12] [SPEAKER_01] Now imagine Russia attacking with, but those were even drones without any explosives in them. Imagine Russia attacking with 50 actual Shahads. With 50 Shahads. So they're attacking an airport in Jeshu, a city where close to Ukraine, lots of Ukraine equipment is being time-bred.

[00:22:31] [SPEAKER_04] Well, that's, you know, when you're talking...

[00:22:33] [SPEAKER_01] Russians say, you know, it's a legitimate target. I attack that. What will Europe and NATO do? Will we send 50 storm shadows to some obscure Russian city? Well, most people I talk to in the EU say that we won't have the guts to do that. But even if, will it be sort of an eye for an eye? No.

[00:23:01] [SPEAKER_01] It will be huge defeat for Europe, huge defeat for the free world. It will be a precedent. And Russia doesn't care. Ukraine blows their cities every night. So what? Yeah. And obviously, huge impact on the markets, huge impact on the economy, huge impact on everything. Now, the second scenario is Ukraine produced about 4 million of these inexpensive FPV drones last year. $500 each.

[00:23:31] [SPEAKER_01] If Ukraine can do that, by the way, United States cannot do it today. You cannot, you can just, there's no capacity to produce that many drones in Europe. But if Ukraine can do for a million, China can easily do for a billion.

[00:23:45] [SPEAKER_02] Yeah.

[00:23:47] [SPEAKER_01] And then, if they really want to, they can probably do them, not with propellers, but with wings. So fixed wing, which is launched from a tube, which flies not 30 kilometers, but maybe 200 kilometers. Yeah. And DGI is the world's leading drone technology company and ensure they have the autonomous capabilities that they need.

[00:24:15] [SPEAKER_01] And then Chinese shipbuilding is also very impressive. So can they build thousands of autonomous submarines capable of carrying those drones? I'm pretty sure they can. So what if China builds 5,000 underwater submarines carrying a couple hundred of these autonomous drones, very inexpensive, capable of going to 200 kilometers inland each,

[00:24:44] [SPEAKER_01] and shows up to the coast of any coast, California or Taiwan or Great Britain? And is there a good response to that? I don't think so. Hunter subs won't be enough. No. Like, that's, that's a very interesting question.

[00:25:05] [SPEAKER_04] Yeah. One of my questions that I've been asking for, why hasn't that already happened? Because Russia does not have very strong AI research, but this is not cutting edge research. This is pretty much a solved problem.

[00:25:28] [SPEAKER_01] Well, that's what I thought. And you would think that... But it's not about only Russia, like even the United States doesn't have the capability. It doesn't have full autonomous drones, which don't rely on GPS. And it doesn't have them at an inexpensive cost. And what it has, well, I'm sorry to say this, but I'm not sure if it works.

[00:25:52] [SPEAKER_00] Because I don't have tremendous respect for companies like Candrel. Yeah.

[00:26:02] [SPEAKER_01] But my only question to these companies is, if you're so great and powerful, why aren't your asses helping us to try that Russia? Why aren't they on the front lines?

[00:26:17] [SPEAKER_04] Although this is what interests me about Eric's.

[00:26:20] [SPEAKER_01] He is different in that extent because his stuff actually works.

[00:26:25] [SPEAKER_04] Yeah. And my guess is that for him, Ukraine, as you said, there's going to be a rearmament. Ukraine is like a development zone and field testing. Yeah.

[00:26:50] [SPEAKER_01] Yeah. Look, I think Eric is a remarkably smart guy. I had the chance to meet him a couple of times. And he just saw around the corner before everyone else. Remember, he was doing work for the US Department of Defense before Russia made Ukraine. Yeah. And he also had the luxury of a lot more free time than many others.

[00:27:20] Yeah.

[00:27:20] [SPEAKER_01] He was able to think deeply and think ahead about these problems. And that's why he focused so much of his attention on Ukraine. And still is focusing. I think one of the smartest things that he did is actually hiring a lot of people here and very quickly iterating with the actual military units. Because you cannot, you know, like I went through the Wicom Nature program in 2016.

[00:27:48] [SPEAKER_01] And one thing they teach folks to get in is, actually they teach everyone, talk to your customers. Talk to your customers. This is the most important thing that you can do as a company. And well, obviously for Android, the customer is Department of Defense. They're doing their job. For Eric Schmidt, the customer is some future fighters which will fight future wars.

[00:28:16] [SPEAKER_01] And so because the future is currently here. Yeah. The future of 4th place here. He comes here regularly. He has a wide team here, forward deployed engineers working with particular brigades. And that's why they were able to build something that actually works. And that actually saves lives because their essence intersects above our heads. And I'll save lives of infantry by destroying Russian advances.

[00:28:45] [SPEAKER_01] So I think that's why he succeeded. And I think other companies are slowly coming around to that. And there's more and more foreign officers talent coming in here. And that's why it is the defense valley. The Florence of Defense.

[00:29:05] [SPEAKER_04] Yeah. Why do you think he's avoiding press?

[00:29:08] [SPEAKER_01] What's such? Oh, I think it's funny. Because he's been avoiding press. But then from time to time, there are these huge press releases, which is like, this is so inconsistent with that strategy. Like the first one was when, I think the last year in Denmark, they signed like a big deal with Zelensky. And there were like photos of Eric and Zelensky.

[00:29:37] [SPEAKER_01] And it was like out of the blue. I saw that. It was like, okay, strategy change. And then again, like they went back into silence. And then I think there was news of them doing a deal with Poland. And so that was again, sort of public. So I have no idea. I think it's just, I think there was a good team of lawyers for thinking this way. Yeah.

[00:30:03] [SPEAKER_04] Yeah. On autonomy, I'm trying to understand these levels of autonomy. You know, I worked on the National Security Commission on AI, which Eric was the chairman and was a Pentagon funded. I think you were referring to his work with the, I've forgotten the name, but the outfit in California that was.

[00:30:31] [SPEAKER_01] I think he was like, like either an advisor to the government or something like that on the matters of the defense technologies.

[00:30:39] [SPEAKER_04] Right.

[00:30:40] [SPEAKER_01] Yeah. Even like writing about that along with some accomplished people in the military. Like you can tell he's been thinking deeply about these problems.

[00:30:53] [SPEAKER_04] But they make it very clear when they speak in the United States, that full autonomy is off the table. That, you know, legal autonomous, lethal autonomous weapons that target autonomously is something that they will not do. Well, what if China does it again? Well, that's right. I mean, that's what I say. That's, that's what they say.

[00:31:23] [SPEAKER_04] But everyone knows it's like Iran, Iran, nuclear with, with their enriching uranium. They say, Oh, we're only going to enrich this much for fuel. But to get to the next level, you just run it longer. So yeah, you just flip the switch basically.

[00:31:39] [SPEAKER_01] So, and then, you know, it's, I think that this is made more complicated by the press than it should be. The ethics of AI in defense. Well, let's take an example of an artillery shot. When do you make a decision to strike a target with an artillery shot? Well, in the moment, well, when we pull it for you and you fired a gun.

[00:32:05] [SPEAKER_01] So you cannot pop an artillery shell 500 meters away from the target and, and ask for a human confirmation, whether it is the right target or whether we shot, you know, two kilometers in the wrong direction and then hit the village. You can think of an autonomous drone as an artillery shell with propellers, and you make a decision the moment to launch the moment to give command. So that makes things much, much easier.

[00:32:32] [SPEAKER_01] The second thing I want to say on this topic is that I think like within five to 10 years, it will become unethical to use weapons without AI. The same way as it might become unethical to use cars without AI. Because those are less lethal, and non-AI weapons will be more likely to cross-prolabor on them, not less level.

[00:32:56] [SPEAKER_01] And the third one is we can be good guys, but bad guys are not going to abide by the rules. So we got to have at least the capability to have these systems working not at the speed of a human command chain, but at the speed of the software command chain.

[00:33:18] [SPEAKER_01] And also, you know, there's, there's lots of conversation about AI and defense, but AI is just sort of the latest big thing. What we should be really talking about, I think there's software in defense.

[00:33:44] [SPEAKER_01] And then, like, look, even AI is such a buzzword, right? Alex Matt is what, 15 years old? And like image recognition capabilities are, you know, at least a decade old, and there are more than sort of cases. And a lot of what is called AI for defense is just image recognition. And then, obviously, we're interacting with large language models now as consumers.

[00:34:13] [SPEAKER_01] So people think that, okay, a drone with AI is something like a chat GPT with propellers that can explode. It's not like that at all. Of course. It's far from that. And if you look, if you think about the robotic system in an academical sense, you have sensors in. So you have basically physical, physical data, or accepted by hardware by sensors. When you have three software layers, you have perception, planning, and control. Right.

[00:34:43] [SPEAKER_01] And then you have actuators out. So commands to monitors, which move the thing somewhere, or make things go. And those layers, perception, planning, control, each of them can be done by classical software, lots of actual, literature on that, lots of evidence, lots of robotic systems, which use that. And then each of them can be done by neural nets, or all three of them can be combined into one neural net,

[00:35:10] [SPEAKER_01] or they can be grouped two by neural net, one by classical algorithms. So we're definitely sort of going towards the world where these systems will be able to do control on their own, and some will have some freedom in decision making whether, you know, but the first decision these systems are going to make is,

[00:35:34] [SPEAKER_01] okay, enemy vehicle is hiding behind a building, whether I go around the building from the right side or from the left side. Yeah. And for years ahead, an AI-enabled defense system will be no closer to a Skynet appraising than a warehouse of 155 caliber shells.

[00:36:02] [SPEAKER_01] It's just not there. And then I think before we are at a risk of being wiped out by defense Skynet, we will be many years on the risk of being wiped out by advanced AI developed on servers of one of these big hyperscale firms. Yeah.

[00:36:31] [SPEAKER_01] So in that extent, I think, you know, we should be careful about AI, but first and foremost about sort of AGI. And even with AGI, I'm not as afraid of it in the coming years.

[00:36:45] [SPEAKER_04] So I think that it's more subtle than Skynet or, you know.

[00:36:52] [SPEAKER_01] Yeah, I think that's a good one. And then also sort of raises, goes back to the question of sovereignty. And like people mostly talk about the sovereign supply chains, but there is also sovereign technology and sovereign know-how. I think it was yesterday, it was in the news that Donald Trump said that they might ban tragedy in Germany or something like that. And that basically sends the signal to not just to Germany, but to entire world.

[00:37:22] [SPEAKER_01] You've got to develop your own financial models. And obviously one person that's very happy is Jensen Huang because he's like, that's his message box. It's like every country should have their own sovereign. And like, of course they should buy our chips. Well, he's in very good business. He's very smart.

[00:37:41] [SPEAKER_01] One thing I think about AGI is actually that, I think there's very high probability that to develop AGI, it will have to get into physical bodies. It will have to be, like, these agents will have to be actual robots interacting with the world.

[00:38:06] [SPEAKER_01] Because I think the good analogy I sort of came up with is, it's like the Plato's cave. Like the current LM's are living in the Plato's cave and they're learning about the world from the shadows on the walls, which are what humans wrote about the world. But they cannot experience the world themselves.

[00:38:26] [SPEAKER_02] Yeah.

[00:38:26] [SPEAKER_01] So for them to be able to experience the world, they have to have the buddies and walk out of the cave. And I think it's also like, like there are a number of researchers who are sort of working in that direction. Like Pefe Li, I think her work is very interesting. Yeah. And yeah, I think like some people in the industry share this intuition as well.

[00:38:48] [SPEAKER_04] So on the autonomy, when you're saying the decision is whether or not to launch, but once it's launched, navigation, targeting, and execution are all autonomous. That has to be onboard, that model. You can't be connecting to the cloud.

[00:39:13] [SPEAKER_01] You can have connectivity as an option, right? Right. But you have to be able to have a fail-safe mechanism. Like there is no connectivity. Connectivity can be jammed. The point where it connects to can be destroyed. So you've got to do all of that on the edge.

[00:39:36] [SPEAKER_01] And actually, so you said it starts when it's launched, but it will gradually start even earlier and earlier. So imagine autonomous systems detecting and then launching. Yeah. Right. Then autonomous systems launching systems which will detect.

[00:39:55] [SPEAKER_02] Yeah.

[00:39:55] [SPEAKER_01] So it's like this, this is a Factorio computer game. Yeah. So you just keep automating, automating, automating, automating. Higher, higher.

[00:40:05] [SPEAKER_04] And the drones that you're building, how autonomous are they at this point? Right.

[00:40:10] [SPEAKER_01] So we are building drones, but we also building the autonomy modules as we call them. And sometimes we just even license the manufacturing of these modules, license them out and we license out the software. So we then we just produce the software which the manufacturer of drones installs in their drones. But we do make our own drones as well. It's just like making drones is sort of a more of a commodity.

[00:40:37] [SPEAKER_01] And the really like interesting thing about this is, you know, drone is like a hamburger. So you can make a hamburger in the kitchen. But the really difficult thing is make them adults. So they make a factory which makes the same hamburgers with a very short timeframe. You can make millions of them without onions or with onions, you know, and that's what the Ukraine drone making companies are.

[00:41:02] [SPEAKER_01] But what we ship currently in mass is sort of level one economy for ground interception. And we call a level zero for aerial interception, which is a detector as the AI detector, which helps pilot to see the target, but doesn't really take over the control of the ground.

[00:41:23] [SPEAKER_01] And the other systems we have on various levels of development and deployment, there are five systems which are sort of ready to be fielded and are in the process of early testing by the ML tournaments. And yeah, it is like, as I've said, there are so many like different dimensions to this, right?

[00:41:51] [SPEAKER_01] So it was like same level of capability, but in different platforms or in different domains or, you know, daytime, nighttime camera, et cetera. And obviously, you know, not everything I can talk about. But yeah, it's a very exciting and fast phrase.

[00:42:11] [SPEAKER_01] And then remember, in 2017, Vladimir Putin said that whoever will master AI will rule over the world. I remember, like, just think about how evil that phrase is. Will rule over the world. Will not cure all the diseases. Will not save the humanity.

[00:42:36] [SPEAKER_01] It's like, it could have been like, even though it couldn't, it could have been save the humanity, right? With his KGB eyes. But no, he was very explicit. What he wants to do.

[00:42:48] [SPEAKER_04] That's right. Although, although he hasn't done a very good job of mastering AI.

[00:42:55] [SPEAKER_01] Well, that's just classical Russia. They lost everything they could. They lost their space technology. And like in 2011, when the US sunsetted the shuttle program, like Soyuz operated by Russia, but developed by Ukrainian Kordov. Because that's Kordov's legacy. Kordov is Ukrainian. He basically thought of himself as Ukrainian.

[00:43:21] [SPEAKER_01] The first ever song which was sang in space to Kordov's wish was a Ukrainian song, which literally says, why am I not a Falcon? Why don't I fly? So, basically, when the American program wasn't ready, everyone flew into space on a Ukrainian ship operated by Russians.

[00:43:48] [SPEAKER_01] And then now they have their planes falling apart, and that they were not able to build their own plane. They have huge setbacks in their rocket programs. So, it's a country that's completely destroyed by corruption and incompetence, but still can do a lot of harm.

[00:44:05] [SPEAKER_04] I'm trying to like zero in on autonomy, because some people say, well, the AI isn't there yet. We don't have models that can target precisely, for example, cannot differentiate between a tank and a civilian car. I don't think that's true.

[00:44:25] [SPEAKER_01] Yeah. Like, come on. Like, who believes that? Of course we can. Like, we have that technology.

[00:44:31] [SPEAKER_04] So, why are the drones produced by Ukraine, why aren't they given that level of autonomy? Is it a cost issue or a supply chain issue?

[00:44:45] [SPEAKER_01] I'll ask you a couple other questions in that line. Why aren't drones produced by the US don't have that capability?

[00:44:51] [SPEAKER_04] Well, that's right. But the US is not in an active conflict. Yes.

[00:44:55] [SPEAKER_01] Then why don't we have self-driving cars? But that's a regulatory issue. It's not... Not really. Like, the technology isn't ready. And the reason... It's close. I mean, Waymo operates, it's going to be in the city soon. I know, I know, I know. But it's been closed for what, 10 years? Yeah, 10 years. Well, I mean, Sebastian from started in 2007 and NASA challenge was even before that. Yeah. Yeah. The answer...

[00:45:23] [SPEAKER_01] And then also, why don't we have AI agents all over the internet doing useful work? Why didn't they replace you and me and all of our friends? And that's, I think the answer to all of those questions about why isn't AI more widespread is the edge cases. Yeah. Like thousands and millions of edge cases. Yeah. Which have to be polished out.

[00:45:45] [SPEAKER_01] And the thing with warfare is that if you have, say, an FBD drone as a system that's used by millions, but what is its effectiveness? Well, it's between 10 and 20%. 10 to 20% of these drones which are used actually hit the target and explode when it's going to cause the desired effect.

[00:46:08] [SPEAKER_04] You're talking about the interceptor drones?

[00:46:10] [SPEAKER_01] No, I talk about the frontline.

[00:46:12] [SPEAKER_04] Right. FBD.

[00:46:13] [SPEAKER_01] So that's a low sort of mark to overcome. Right? And that's why drones like ours are currently used in the frontline and they increase the sufficiency by two to four times even with level one autonomous system. And then as you go further with wider, wider autonomy, you basically, you multiply the edge cases. And then add to that, not autonomous, but robotics problems.

[00:46:41] [SPEAKER_01] Like most of the things we deal with is not about AI, is about software and robotics. So from the outside, because AI is such an out buzzword, it might seem that a drone company that does AI has 80% of its staff or engineering staff doing AI. But it's more like 10%. Because there are so many other problems.

[00:47:11] [SPEAKER_01] So it will happen. It will just have to go through all of these issues and it's hard to predict when, because it's always this S-curve, right? You go over some threshold and then you have this takeoff. I certainly predict that it's going to not take as long as it took the Islamic course. Yeah.

[00:47:30] [SPEAKER_04] And regarding the US, I mean, there's an existential imperative in Ukraine that so things are moving much faster than elsewhere in the world. Sure. Do you, in terms of the targeting, navigation has been pretty much solved, right?

[00:47:55] [SPEAKER_01] I don't really, unfortunately. Oh, exactly. Yeah. It's like, again, this is like, there's so much sort of common, like you would think it was solved, but it's not. Because if you think about the navigation problem, well, first of all, it's not one problem. It's like a hundred different problems because navigation for a $500 drone, which has to fly 40 kilometers and maybe 20 kilometers and then return back

[00:48:21] [SPEAKER_01] and flies at speeds of up to 100 kilometers per hour, or 60 miles per hour. That's one problem. Navigation for a wing missile that goes slightly subsonic is another problem. Navigation for, you know, I was like, last night, corrections for hypersonic is a very different problem as well. You're flying above the altitude of flight as very much important for navigation. As you're flying high, it's easier to navigate.

[00:48:49] [SPEAKER_01] If you're flying low, you have less data about what's, what's next to you. Then you fly above a body of water, much more difficult to navigate than a body of land if you fly at low altitude. So all of these factors, the length, the cost, the altitude, the terrain, all of them impact navigational system and their, there will only like one AI, you know, Google Maps that solves it for anything and cost nothing and this is on your smartphone.

[00:49:19] [SPEAKER_01] It's going to be very different than that. Although maybe at some point there will be something like, like I just described. And then, but yeah, you were about to ask about the target detection actually. Yeah.

[00:49:31] [SPEAKER_04] So the target, that part is solved, right? But you have a camera, you have object recognition, either thermal or segmented. And you, the drone hits that target. I'm talking about kamikaze drones, right? It hits that target.

[00:49:52] [SPEAKER_02] Yeah.

[00:49:53] [SPEAKER_04] Why isn't, or is that already being used?

[00:49:57] [SPEAKER_01] Well, it is, it is solved to some extent, right? It's, it's easy to spot, you know, a green tank on, on, on, on a white, snowy field. It's much more difficult to spot something camouflaged. Then you see a Russians riding on donkeys. Like was that in the data set? Probably not. Then you see the Russians dressed up like penguins.

[00:50:26] [SPEAKER_01] Was that in the data set? Also not really. Then, you know, when you talk about the practicality of this, it's not just about the algorithm that can recognize an image. Say you have an FPV drone that's autonomous. How fly, how high should it fly? If it flies slower, it, it's closer to what it has to recognize.

[00:50:56] [SPEAKER_01] It can maybe recognize it better. But it has a smaller field of view. And how fast does it fly? Does it, does it scout the territory in order to find something? Or how does the recognition integrate with navigation? And then how does that integrate with the tactics of its usage? Like do we need autonomous drones against any infantry? Or do we need any logistics?

[00:51:22] [SPEAKER_01] There are so many of these questions that in practice, going from the technology of being able to recognize something to the device that's actually usable, useful, usable, and creates more utility than the existing alternatives is not a super obvious thing.

[00:51:44] [SPEAKER_04] Yeah. Are you worried about fully autonomous just in terms of the human cost of war?

[00:51:53] [SPEAKER_01] I'm not sure what you mean by that question.

[00:51:56] [SPEAKER_04] Well, if you have fully autonomous AI guided fire and forget drones, maybe even launched.

[00:52:07] [SPEAKER_00] Yeah.

[00:52:09] [SPEAKER_01] That's a pretty terrifying prospect on both sides. Oh, no. Like it's terrifying if the enemy has it. If we have it, it's a savior. Yeah, that's right. And if we don't have it, that is what is terrifying. Right? Yeah. And like you obviously you need to have autonomous interceptors to counteract enemy autonomous drones. Because the mass and like how many of them they can deploy will not be as intended by humans now.

[00:52:38] [SPEAKER_01] And then obviously, even now, it's hard for humans to match the speed, agility of machines. That will become harder and harder. So what I'm terrified about is that we won't get there as fast as the enemy does.

[00:52:59] [SPEAKER_04] Yeah.

[00:53:00] [SPEAKER_01] That's un-terrified effect. Yeah.

[00:53:02] [SPEAKER_04] Okay.