This new AI technology enhances video analysis by detecting human actions in real time
- A new AI capable of recognizing complex visual data has been developed
- SMAST can learn and predict complex human actions
- The tool could be used in surveillance, healthcare, and autonomous driving, researchers say
Researchers at the University of Virginia’s School of Engineering and Applied Science have taken AI’s visual data capabilities a step further with their latest innovation – an AI-driven video analyzer called the Semantic and Motion-Aware Spatiotemporal Transformer Network (SMAST).
This system offers precision in detecting human actions, promising applications in areas like public safety, motion tracking, and even autonomous vehicle navigation.
At the heart of SMAST’s capabilities is its ability to process complex video footage by focusing on the most relevant parts of a scene.
The system integrates a multi-feature selective attention model and a motion-aware 2D positional encoding algorithm. These features work in tandem to ensure that the AI can accurately detect and interpret human actions.
The selective attention model allows SMAST to concentrate on crucial elements, such as a person or an object in motion, while ignoring irrelevant details. For instance, it can distinguish someone throwing a ball from someone simply raising their arm.
Meanwhile, the motion-aware algorithm enables the AI to track movements over time, remembering how objects and people have shifted within a scene. This gives SMAST the ability to comprehend the relationships between different actions, making it more effective at recognizing complex behaviors.
In the security and surveillance sectors, the SMAST system can enhance public safety by detecting potential threats in real-time. For example, it can identify suspicious behavior in a crowded space or recognize if someone is in distress. In health care, the technology could be used to track patients’ movements, enabling better motion analysis for rehabilitation or monitoring during surgery.
The researchers claim SMAST stands out in its ability to handle chaotic, unedited footage. SMAST’s AI-driven approach apparently allows it to learn from data, adapting to various environments and improving its action detection capabilities. The tool has been subjected to several academic benchmarks including AVA, UCF101-24, and EPIC-Kitchens and it did quite well.
“This AI technology opens doors for real-time action detection in some of the most demanding environments,” said professor and chair of the Department of Electrical and Computer Engineering, Scott T. Acton. “It’s the kind of advancement that can help prevent accidents, improve diagnostics and even save lives.”
Via TechXplore
You might also like
A new AI capable of recognizing complex visual data has been developed SMAST can learn and predict complex human actions The tool could be used in surveillance, healthcare, and autonomous driving, researchers say Researchers at the University of Virginia’s School of Engineering and Applied Science have taken AI’s visual data…
