As we learned in the previous post Edge AI is particularly useful where there is a lot of sensor data and limited compute functionality. In a city, it may seem like we have all the bandwidth available but 50% of the earth is untouched by human development, a very small % of the earth has cellular and wireless coverage and many places do not have power. Hence there is a lot of potential for battery-powered edge devices to give us access to these places.
Sensors are becoming cheaper, more sophisticated, and less power-hungry. For example, modern AI voice assistants and cell phones have low-power digital signal processing hardware so that they can continuously listen for the wake word like “Alexa”, “Ok Google” or “Siri”. The hardware is chosen with specific wake word detection only and does not drain the battery. At other times the same functionality might need to be built in existing hardware (brownfield instead of greenfield). In this scenario, one would have to look at using the spare cycles in the device’s existing embedded processor. A firmware upgrade on the device can add new functionality.
Real World Products
Forest Fires
Understanding and controlling systems – Equipment failure across vast swathes of forests and wilderness can be dangerous to animals and vegetation. Sensors at each electrical pylon can measure temperature, inclination, and voltage, combining this information with deep learning to identify a fault that might develop. The maintenance crew can fix the issue before it becomes a fire hazard. This device needs to be i) rugged to withstand the elements of nature. ii) It should only be able to transmit faults (not more than 250kb per month), not regular data, to iii) conserve battery (it cannot run on power since it is reporting power 🙂 and bandwidth. With Edge intelligence, the battery can last for 20 years. If we were to use power and switch to battery when the power line is down then that would include adding more hardware which might not be required.
BLERP Analysis – Bandwidth (connectivity is limited since it is a remote location), Latency (should be able to report a fault as soon as it occurs), Economics (no unnecessary communication and runs on battery), Reliability (runs on battery that can last for 20 years), Privacy (non issue)
Elephant Tracking
Understanding and controlling living beings – Poaching, interactions with humans in search of food, disease, and pressure on the natural habitat have led to a rapid decline in the elephant population. ElephantEdge is an initiative to design a collar that helps in understanding elephant behavior. It can give an insight into the location, movement, and health of the animal that can be further analyzed with ML models.
BLERP Analysis – Bandwidth (Connectivity very limited – depends on LoRa-equipped satellites that pass once a day), Latency (once a day), Economics (replaces laborious traditional methods for monitoring elephants), Reliability ( 5 years battery life), Privacy (not a big issue)
Smart Packaging in Ships
Keeping track of Objects – Products travel thousands of miles in a ship before they can get to the customer. For high-value items, various sensors can be attached to the shipment so that it arrives in the best possible shape. If there is any issue then it will become easier to identify with the logs. These logs could be periodically uploaded if there is connectivity else they can analyzed on the ship and the crew can make necessary changes or the logs can be analyzed on arrival to see if anything went wrong. For example, if the accelerometer rating of a shipment shows variations then it could suggest rough handling or a fall that might compromise the shipment.
BLERP Analysis – Bandwidth (connectivity will be an issue so most data might stay on the device), Latency (not required), Economics (it is expensive to transmit data when the ship is en route), Reliability (device should be able to store data and have enough battery to hold data), Privacy (not an issue)
Blurring Background during Remote Work
Generating or Transforming Signal – To maintain privacy, the video calling tools allow to blur the background of the video stream, while leaving the subject sharp and intact. The technique used is called segmentation which uses deep learning to identify one category of pixels different from another category. To ensure the privacy of the subject, it is essential that this computation is done on the edge else it will result in a non-blurred background being passed onto the central servers.
BLERP Analysis – Bandwidth (needs to happen on the original high-quality video stream without sending that out, so needs to happen on the edge), Latency (there is no extra latency of sending it out to another server, it needs to be real-time and hence on the edge), Economics (cheaper to do it on the device), Reliability (simpler and reliable when done on the device), Privacy (since the stream is augmented on the device hence nobody would see the original stream)
Other Types of Applications
As shown in the examples above, applications can broadly fall into
- Keeping track of objects – container ships to movement of objects in a warehouse. Smart packaging with humidity and temperature controllers to count products on shelves in a retail store (vision sensor), analyze the movement of waste in an ocean, track obstacles at sea to avoid collisions, and earth analysis for buried resources. Cheap edge AI sensors making use of opportunistic connectivity can provide gap analysis and easy monitoring.
- Understanding and controlling systems – Production lines, transport networks, climate control, and smart home appliances. Monitoring oil rigs for maintenance, autonomous driving of farm and commercial equipment, shaping traffic flow on busy highways, faulty items on production line, smart vacuums, thermostats
- Understanding and controlling living beings – Bridging the gap between biology and technology. Alerting workers in dangerous environments, analyzing medical trends to alert healthcare professionals, alerting agencies on identification of an unknown person on a property, recognizing voice commands to control an appliance, number of people in a hall or bus stop to dispatch additional busses, warning driver if they fall asleep on the wheel, etc. Identifying wildlife, crop diseases, elephant edge, and cooking to perfection – This is a category where privacy is important so the data needs to be on the device in many cases to be accepted.
Deep-learning vision models can be trained to accept certain images and reject others. For example, if we want to identify only elephants and ignore other wildlife, then the model would be trained that way. Now, only the data related to elephants would be transmitted. The cameras can be equipped with low-power or cellular radio transmitters to transmit data. - Generating and transforming signals – This is primarily Digital Signal Processing. This means working on the input signal to alter it or decorate it further. For example, removing background noise from an audio signal, blurring the background of the video feed during calls, generating human speech from text, translating text from one language to another, and transcribing spoken language into text.
All these are examples where one signal is converted into another.
Summary
In this post, we saw examples of 4 major categories of applications where Edge AI can be used. From Objects to living beings to controlling systems to generating and transforming signals.