Teleoperation of Heavy Machinery? The next generation of remote-control plant machinery and vehicles


AdobeStock_228815038 2

Is the concept of Remote Driving Machinery the same as Remote Driving Vehicles?

Remote driving is not to be confused with self-driving. In remote driving the vehicle is driven in the same way as normal by a human driver, it just happens to be in a car vehicle simulator some distance away.  Self-driving is where the car decides for itself what it should do, and this is sometimes known as autonomous driving. The same principle could be applied to heavy plant machinery such as earth movers, diggers and cranes but there is more to these machines than just driving – they need to be operated, albeit remotely.

Why should I use remote driving over self-driving?

Imagine situations in hazardous areas where special instructions are required that a machine cannot determine. Maybe machinery needs to be manoeuvred around obstacles that its sensors cannot understand. There are many hazardous areas that could are deemed high risk to operate in and requires human logic to view in real time, decipher and determine the best cause of action. Maybe an area that has been subject to an earthquake with many unseen hazards. Or an avalanche area that needs clearing, or even a minefield in a warzone where a mine clearance machine needs to be remotely operated.  Because of the dangers involved, it would be too dangerous to put people in harm’s way. If vehicles can be dropped in a hazardous area that are not fit for humans where they can operate, or even deliver and return goods, safely then they will need to be remotely operated in real time. This would not only apply to heavy remote control of plant material, but to any vehicle, whether it be cars’, buses or heavy goods vehicles such as lorries and trucks.

The ability to operate remote heavy machinery remotely in the same manner can be overcome by not having to put a human operator in the driving seat within a dangerous or hostile environment. There are many hazardous areas where you may not want to expose people. For example, in a war zone where there are booby traps or potential snipers, or a bulldozer or excavator having to dig in biologically hazardous or radioactive areas, or an old military site where potential bombs are likely to have been discarded. Even excavating in an unstable landscape caused by simple geology, utilizing a remote operator would be a better alternative than putting the driver’s life at risk.

Even where areas are not hazardous, remote operators can be used where there is a potential shortage of skilled operators. A single operator would not need to travel between different sites and could potentially do many jobs in one day from a single location. People would not need to travel between sites at great expense – only the machinery would need to be delivered.

What is the technology behind remote driving?

The one major overriding factor is safety. In order to facilitate remote driving cars or heavy machinery, the remote operator needs to have a real time view with almost no latency. The remote operator is known as a teleoperator and often use a vehicle simulator and they need to control and operate the machine with no delay.

For example, imagine a car pulling into a main road, the view the driver observes must be in real time, if there is even a one second delay to a remote operator who then decided to pull out as he observes a delayed view, then the consequence could be catastrophic. Much can happen in a single second on the road. The same can be said of controlling heavy machinery in a hazardous environment.

Soliton Systems, a Japanese manufacturer of live streaming solutions, has been working with several manufactures of vehicles, machinery and cranes to deliver a safe remote-control solution that can be used by a teleoperator.  This includes a major automobile company in Japan to deliver a remote driving car vehicle that can be driven by a remote operator.

Remote driving

                                                      Automatically Generated Cockpit Screens

  • To live stream video with super low latency to the vehicle simulators
  • To live stream reliably over untethered cellular networks securely even in challenging network conditions
  • To have a return network path for driving remote vehicle for the remote driving operations

Soliton has a legacy in providing mobile live streaming video solutions that are used by many global broadcasters for live news gathering, live sports production, and encrypted mobile surveillance for use with law enforcement.

With their Zao product, they were the first to market with a mobile H.265 HEVC encoder that could live stream full HD video over multiple 4G connections simultaneously. And they were the first to offer a solution with a latency as low as 240ms from camera to the remote receiving platform. This latency is unheard of with mobile encoders over 4G and is even more incredible when there is AES256 encryption involved.

In terms of ensuring reliability, it utilizes a bonding technique over 4G known as RASCOW. The Zao can utilize up to 6 SIM cards that can utilize different network operators and bonds them together as a single connection for reliability and bandwidth. At the receiving end the video is reassembled but this technique, coupled with H.265, provides a very reliable stream even if some networks disappear or are in contention with many users such as at a well-attended public event. RASCOW can optimize the video stream even if there is very low signal on all the network operators ensuring video is delivered even in the most challenging of network conditions.

When it comes to reliability, this is managed via RASCOW but to overcome the challenges of remote teleoperation of vehicles and heavy plant machinery, Soliton and their customer needed a product that could provide even lower latency to stream video than even their own ground breaking Zao can offer at 240ms.

Introducing the Zao-SH. Only weighing 400 grams and supporting up to 3 bonded 4G networks plus a LAN connection it is the lightest and smallest mobile encoder on the market for live streaming with a unique focus on ultra-low latency. And by ultra-low latency, it is a figure never achieved in a mobile encoder. Over a LAN the glass-to-glass latency (from the camera to the video output in the car simulator) was measured at 35ms including H.265 encoding, transmission and decoding. Over 4G, this was as low as 65ms. This is an unprecedented figure. With 5G, this ultra-low latency is reduced even further.

For the vehicle and plant control, the remote simulator converts the outputs from the steering, accelerating, breaking and other operational functions into a digital stream that is sent back via the same mobile network,  either 4G LTE and/or 5G, with a return path back to the Zao-SH, where the output is converted into a series of hydraulic commands that are used to operate the vehicle via a series of servo accentuators.

The aim of the Zao-SH to deliver ultra-low latency encrypted video streaming with full high-quality HD video in a reliable manner, with a return path for remote control was achieved.

If you wish to understand more about what Soliton Systems can do for you in terms of remote driving or remote machinery operation, then please contact us for more information.

 

Anita Ghosh

Subscribe Here!