“Any sufficiently advanced technology is indistinguishable from magic.” – Arthur C. Clarke
History shows us that the graphical processor unit (GPU) appeared only after the public consumption of software requiring graphical acceleration was surpassing the computer processing unit’s (CPU) capabilities; in video games, for example.
Today, we are looking at a similar phenomenon with imaging capabilities. The use of smartphones for photography has made it increasingly difficult to address innovation in computational imaging or computer vision while keeping up with the tight requirements of power consumption.
If the CPU is at the core of any device, and the GPU is the best way to address graphic-related use cases, it’s only logical to conclude that the image processing unit (IPU) is the future of computational imaging on mobile devices.
The IPU was created with hardware-software hybrid implementation in mind, to raise the bar for all existing and future imaging technologies by achieving higher performance, shorter execution times and the lowest power consumption possible. It is not enough to have a high quality software implementation if after 10 minutes of taking photos users can’t answer a call because their phone is burning up.
A typical IPU configuration would be around 1.5 sq mm , 28 nm technology and it would consume less than 80 mW of power. It would be capable of handling long range object detection (faces at all angles, full and half body concurrently), coarse optical flow frame to frame registration, bi-cubic re-sampling or correction (lens + video image stabilization) 4k @ 60 FPS, hybrid biometrics (Iris and Face Recognition) and support many other computer vision and computational photography features.
MAIN IPU COMPONENTS
Programmable processing engine ● Hardware acceleration engines ● Smart cache
Keep local data local thus reducing the bandwidth
Fast interaction between software and hardware modules
Customized architecture for image processing data flow
Power savings achieved by solving difficult tasks directly in hardware
Easy to fine tune algorithms in software
LARGE MEMORY BANDWIDTH REQUIREMENTS
Generated by an increase in requirements for larger resolutions and larger frame rates, as well as additional requirements for image analysis and enhancements like Face Detection, Body Detection or Distortion Correction.
Pixel frame rate increase drives more processing power requirements and power consumption. With a general purpose CPU / DSP it is impractical to do input image scanning, feature extraction and object matching with multiple scales and rotations.
A compromise has to be made between performance and required resources for implementation.
Hardware processing of pixels on-the-fly as they are ready from the ISP to the main memory.
ANALYTICS ON SMALL INPUT RESOLUTION IMAGES
Hardware-software hybrid implementation offers the best mix of price, performance, power consumption and flexibility.
Wide range of core technologies are supported.
Cost-effective and future-proof, solutions are scalable with an increase in resolutions and frame rates.
DRONES & ROVs
INTERNET OF THINGS