“Any sufficiently advanced technology is indistinguishable from magic.” – Arthur C. Clarke

OVERVIEW

ARCHITECTURE

APPROACH

ENGINES

MARKETS

OVERVIEW

History shows us that the graphical processor unit (GPU) appeared only after the public consumption of software requiring graphical acceleration was surpassing the computer processing unit’s (CPU) capabilities; in video games, for example.

Today, we are looking at a similar phenomenon with imaging capabilities. The use of smartphones for photography has made it increasingly difficult to address innovation in computational imaging or computer vision while keeping up with the tight requirements of power consumption.

If the CPU is at the core of any device, and the GPU is the best way to address graphic-related use cases, it’s only logical to conclude that the image processing unit (IPU) is the future of computational imaging on mobile devices.

The IPU was created with hardware-software hybrid implementation in mind, to raise the bar for all existing and future imaging technologies by achieving  higher performance, shorter execution times and the lowest power consumption possible. It is not enough to have a high quality software implementation if after 10 minutes of taking photos users can’t answer a call because their phone is burning up.

A typical IPU configuration would be around 1.5 sq mm , 28 nm technology and it would consume less than 80 mW of power. It would be capable of handling long range object detection (faces at all angles, full and half body concurrently), coarse optical flow frame to frame registration, bi-cubic re-sampling or correction (lens + video image stabilization) 4k @ 60 FPS, hybrid biometrics (Iris and Face Recognition) and support many other computer vision and computational photography features.

ARCHITECTURE

MAIN IPU COMPONENTS

Programmable processing engine Hardware acceleration engines Smart cache

IMPLEMENTATION HIGHLIGHTS

Keep local data local thus reducing the bandwidth

Fast interaction between software and hardware modules

Customized architecture for image processing data flow

Power savings achieved by solving difficult tasks directly in hardware

Easy to fine tune algorithms in software

APPROACH

  COMMON PROBLEMS

LARGE MEMORY BANDWIDTH REQUIREMENTS

Generated by an increase in requirements for larger resolutions and larger frame rates, as well as additional requirements for image analysis and enhancements like Face Detection, Body Detection or Distortion Correction.

POWER CONSUMPTION

Pixel frame rate increase drives more processing power requirements and power consumption. With a general purpose CPU / DSP it is impractical to do input image scanning, feature extraction and object matching with multiple scales and rotations.

COMPROMISED FLEXIBILITY

A compromise has to be made between performance and required resources for implementation.

  OUR SOLUTIONS  

ON-THE-FLY-PROCESSING

Hardware processing of pixels on-the-fly as they are ready from the ISP to the main memory.

ANALYTICS ON SMALL INPUT RESOLUTION IMAGES

Hardware-software hybrid implementation offers the best mix of price, performance, power consumption and flexibility.

MULTI USE

Wide range of core technologies are supported.

SCALABLE SOLUTIONS

Cost-effective and future-proof, solutions are scalable with an increase in resolutions and frame rates.

MARKETS

SMARTPHONES

LAPTOPS

TABLETS

ACTION CAMERAS

DIGITAL CAMERAS

VIRTUAL REALITY

WEARABLES

BIOMETRICS

SURVEILLANCE

DRONES & ROVs

AUTOMOTIVE

INTERNET OF THINGS