How to drive a vehicle or heavy machinery when you cannot be there

June 22, 2020

AdobeStock_228815038 2The answer to remote driving vehicles and remote heavy machinery operation

Remote driving is not to be confused with self-driving. In remote driving the vehicle is driven in the same way as normal by a human driver, it just happens to be in a car simulator some distance away.  Self-driving is where the car decides for itself what it should do.

Why should I use remote driving over self-driving?

Imagine situations in hazardous areas where special instructions are required that a machine cannot determine. Maybe a car needs to be manoeuvred around obstacles that its sensors cannot understand. Perhaps through a minefield, or maybe where there are external influences that a car would not know about such as an impending tsunami or avalanche that needs to determine how, where and when a vehicle needs to be routed. Because of the dangers involved, it could be too dangerous to put people in harm’s way, or vehicles have been dropped in a hazardous area that are not fit for humans that need to deliver and return goods. With recent Coronavirus situations, this is no longer the realm of sci-fi and movie scenarios. This could be applied to any vehicle, whether it be cars’, busses or heavy goods vehicles such as lorries and trucks.

Not only driving, the ability to operate remote heavy machinery remotely in the same manner can overcome having to put a human operator in the driving seat within a dangerous or hostile environment. For example, in a war zone where there are booby traps or potential snipers, or a bulldozer or excavator having to dig in biologically hazardous area, or an old military site where potential bombs are likely to be or have been discovered. Even excavating in an unstable landscape, a remote operator would be a better alternative than putting the driver’s life at risk.

Even where areas are not hazardous, remote operators can be used where there is a potential shortage of skilled operators. A single operator would not need to travel between different sites and could potentially do many jobs in one day from a single location. They would not need to travel between sites and lose time – only the machinery would need to be delivered.

What is the technology behind remote driving?

The one major overriding factor is safety. In order to facilitate remote driving cars or heavy machinery, the operator needs to have a real time view with almost no latency. Imagine a car pulling into a main road, the view the driver observes must be in real time, if there is even a one second delay and the car pulls out, a lot can happen in a second on main road, and the consequence could be catastrophic.

Soliton Systems, a Japanese manufacturer of live streaming solutions,  has been working with a major automobile company in Japan to deliver a remote driving car vehicle that can be driven by a remote operator.

Remote driving

                                                      Automatically Generated Cockpit Screens

The major challenges are as follows: 

  • To live stream with super low latency to the simulator
  • To live stream reliably and securely even in testing network conditions
  • To have a return network path to the car for robotic driving commands

Soliton has a legacy in providing mobile live streaming video solutions for use in many global broadcasters for live news gathering and reporting, live sports production and encrypted mobile surveillance for use with law enforcement.

With their Zao product, they were the first to market with a mobile H.265 HEVC encoder that could live stream full HD video over multiple 4G connections simultaneously. And they were the first to market to offer a solution with a latency as low as 240ms from camera to the remote receiving platform. This latency is unheard of with mobile encoders over 4G and is even more incredible when there is AES256 encryption involved.

In terms of ensuring reliability, it utilizes a bonding technique over 4G known as RASCOW. The Zao can utilize up to 6 SIM cards that can utilize different network operators and bonds them together for reliability and bandwidth. Portions of video is sent across each mobile operator depending on how much bandwidth is available in real time. At the receiving end the video is reassembled but this technique, coupled with H.265, provides a very reliable stream even if some networks are low or are in contention with a lot of users. RASCOW can optimize the video stream even if there is very low signal on all the network operators ensuring video is delivered even in the most challenging of network conditions.

So reliability is managed via RASCOW but to overcome the challenges of remote driving, Soliton and their customer needed a product that could provide even lower latency to stream video than even their own ground breaking Zao can offer at 240ms.

Introducing the Zao-SH. The Zao-SH started life as the Zao-S which was developed after the Zao as a smaller version, only weighing 400 grams and supporting up to 3 bonded 4G networks plus a LAN connection. It then evolved into the Zao-SH with a focus on ultra-low latency. And by ultra low latency, it is a figure never achieved in a mobile encoder. Over a LAN the glass-to-glass latency (from the camera to the video output in the car simulator) was measured at 35ms including H.265 encoding, transmission and decoding. Over 4G, this was as low as 65ms. This is an unprecedented figure. When 5G arrives on mass, the expectation is that this ultra-low latency will reduce even further.

For the car control, the simulator converts the outputs in the simulator from the steering, accelerating, breaking and other car functions into a digital stream that is sent back via the same network, that is over the 4G network, back to the Zao-SH, where the output is converted into a series of hydraulic commands that are used to operate the car via a series of servo accentuators.

The aim of the Zao-SH to deliver ultra-low latency encrypted video streaming with full high quality HD video in a reliable manner was achieved.

If you wish to understand more about what Soliton can do for you in terms of remote driving or remote machinery operation, then please contact us for more information

Get our newsletter direct to your inbox