Tag: Google

Ever since it introduced Android, Google has struggled to provide a camera framework for developers that allows them to both build camera-enabled apps quickly and take advantage of the advanced capabilities being offered by phone makers. Its first Camera API was limited, and the second version is complicated. Neither has a vendor-independent way to activate some of the advanced modes that have been added to phones for improving image quality and adding stylistic effects. Now, Google is launching CameraX, a library that provides not just a simplified way for developers to access Android’s Camera2 functionality, but provides extensions for additional capabilities.

CameraX Is a Simpler Way to Harness the Power of Camera2

CameraX is provided as a Jetpack support library, and the basic portion of its capability is usable on Android OS versions back to Android 5.0 (API level 21). Its wrappers provide a use-case-centric set of interfaces to Camera2, and add lifecycle awareness to help reduce programming overhead. Google also says it reduces device dependencies, so CameraX code should run across all types of hardware (there are also calls to ask whether a device has a particular capability like a front camera).

To work with CameraX a developer specifies a desired use case with configuration options. Listeners are then added to handle the data output by the CameraX library — which can be either in the form of a data stream or written directly to a file. Finally, the use cases are bound to Android Architecture Lifecycles, so that CameraX can handle some of the housekeeping associated with setup and teardown of resources for the application. One nice feature of CameraX is that multiple use cases can be run simultaneously, so a preview can remain live while images or video are analyzed and perhaps captured, for example.

CameraX supports several use cases: Preview, to get an image on the display; Image analysis, to get direct access to the image buffer; and Image capture, to save a fully-processed image or video. Google provides several sample applications for various use cases. As is the recent trend, the ones I looked at are written in Kotlin with alternate Java versions.

After a few lines of setup, CameraX commands are fairly straightforward. For example, to simply capture an image from the camera, there is a takePicture() method. Prior to calling it, another simple call lets an application select which camera to use, or to set other parameters. As you’d expect, CameraX requires the app to have CAMERA permissions, and WRITE_EXTERNAL_STORAGE permission on versions of Android prior to Q if you want to capture directly to the file system.

Extending RAW Image Capture to Multi-frame Scenarios

It was a big advance in capability when Android phones started offering the option to save RAW images for later post-processing, instead of only allowing access to their pre-processed JPEG versions. But with smartphones increasingly relying on the sophisticated merging of multiple frames to create a single output, post-processing a single RAW frame isn’t always the best option. With CameraX, app developers can directly read the stream of frames coming from the camera. That’s important for applications like machine learning and artificial reality (AR). However, it is also a potential boon for those wanting to provide their own image processing pipelines — either on the device or later on a desktop computer or in the cloud.

CameraX Offers Extensions for HDR, Night, Portrait, and Beauty

One area that has made third-party camera apps less attractive is the difficulty they have harnessing the advanced computational imaging capabilities of newer-model smartphones. For devices that support CameraX extensions, applications can access their advanced modes including HDR, Night photography, Portrait mode, and Beauty enhancements with the simple addition of a few lines of code to an existing Camera2 application.

For an app to have access to an extension, the phone maker needs to add a hook to the CameraX library to the vendor’s own API. If a vendor doesn’t provide an implementation of a capability, CameraX just reports it as being unavailable. Currently, developers using the alpha version of CameraX can make use of extensions on the following phones:

Samsung (HDR, Night, Beauty, Auto): Galaxy Note 10 series (pictured, top)
(demonstrated at Samsung’s SDC19)

Huawei (HDR, Portrait): Mate 20 series, P30 series, Honor Magic 2, Honor View 20

CameraX Rollout

CameraX is currently in alpha, but it’s expected to get to beta status — meaning final APIs according to Google — in December. Samsung showcased its support for CameraX this week at its Software Developer Conference, and it co-hosted a session with Google on how developers can take advantage of both CameraX and its extensions on the latest Samsung phones.