Comment on page

Multiview Image and Decoding Pipeline

Learn about the Multiview Image and decoding different file types to generate multiple views.

In the previous sections, you learnt how to leverage the SDKs to display different image in 4V using a Quad, as well as convert other supported image formats to a Quad using the built in Image decoders and synthesizers. 
In this section, you will learn about the decoding process, and how to leverage the SDK to perform operations on the different image formats in your android applications.
What is a Multiview Image?
The MultiviewImage is a collection of ViewPoints, and global metadata that affects all views like gain, convergence and depth of field. It is a decoded representation of an image that can be rendered on a Leia Lightfield display. 
A ViewPoint represents the image and disparity map from a single camera view point.
Decoding Pipeline
The Multiview Image Decoder can be used to decode the different image formats supported by the SDK and obtain a Multiview Image instance.
The Multivew Image provides a unified API for loading the different image formats. 
The Android Media SDK provides a set of operators that work with the Multiview image, allowing you to perform operations like View synthesis, disparity estimation and depth of field.
The Multiview Image can be displayed on the screen using the QuadView as we seen in the previous sections. 
Multiview Image Decoder
The MultiviewImage decoder decodes different image types to return a MultiviewImage instance. 
Using the Multiview Image Decoder 
You can obtain an instance of the MultiviewImageDecoder by calling MultiviewImageDecoder.getDefault()
Kotlin
Java
val multiviewImageDecoder = MultiviewImageDecoder.getDefault()
MultiviewImageDecoder decoder = MultiviewImageDecoder.getDefault();
Decoding Images
The MultiviewImageDecoder provides different methods to decode images. In the previous sections, we decoded a LIF image using raw byte[] as well as decoded a SBS image by using the File Uri.
1. Decoding with Uri
This is the most recommended method for decoding the different image formats due to its compatibility  with all image formats. 
You can call the following function to decode using the file Uri. 
Kotlin
Java
val multiviewImage = multiviewImageDecoder.decode(context, fileUri, 1280 * 720)
MultiviewImage multiviewImage = multiviewImageDecoder.decode(context, fileUri, 1280 * 720);
The function takes in 3 parameters-
The Application context.
The File Uri or contentUri for the image on the device's internal storage.
The output resolution for all the decoded viewpoints in the Multiview image. This rescales all views to the set number of pixels, if necessary.
Optionally, if you know the encoded image file contains ViewPoints with the same resolution, you can decode without specifying the output pixels. 
Kotlin
Java
val multiviewImage = multiviewImageDecoder.decode(context, fileUri)
MultiviewImage multiviewImage = multiviewImageDecoder.decode(context, fileUri);
Note: Calling decode() without the output image resolution will throw an Exception if all decoded viewpoints are not the same resolution.
2. Decoding with raw bytes
You can also decode images using raw bytes. However, this method is only supported for Leia Image file types. (H4VBinary & H4V_XMP)
Kotlin
Java
val multiviewImage = multiviewImageDecoder.decode(imageBytes, 1280 * 720)
MultiviewImage multiviewImage = multiviewImageDecoder.decode(imageBytes, 1280 * 720);
Optionally, if you know the encoded image file contains ViewPoints of the same resolution, you can decode without specifying the output pixels. 
Kotlin
Java
val multiviewImage = multiviewImageDecoder.decode(imageBytes)
MultiviewImage multiviewImage = multiviewImageDecoder.decode(imageBytes);
Note: Calling decode() without output image resolution will throw an Exception if all decoded viewpoints are not the same resolution.
Detect File Types 
You can also use the MultiviewImageDecoder to detect the image file type. 
Kotlin
Java
val decoder = MultiviewImageDecoder.getDefault()
val fileType = decoder.getFileType(context, fileUri)
MultiviewImageDecoder decoder = MultiviewImageDecoder.getDefault();
MultiviewFileType fileType = decoder.getFileType(context, fileUri);
This function takes in the File Uri and returns a MultiviewFileType. The returned Multiview File types are categorized below
Multiview File Type
Image File Type
MultiviewFileType.H4V_BINARY
​Leia Image Format
MultiviewFileType.H4V_XMP
​Leia Image Format
MultiviewFileType.TWO_BY_ONE
​SBS (2x1)​
MultiviewFileType.TWO_BY_TWO
​Quad (2x2)​
The decoder returns null if the Uri specified is not a supported file type.
Decoder Compatibility
To ensure decoding compatibility for SBS (2x1) and Quad (2x2) images, you need to make sure the files for these image formats follow proper naming conventions. 
SBS image file names must have a suffix of _2x1
Quad image file names must have a suffix of _2x2
File Type
Valid File name
SBS (2x1)
cards_2x1.jpg
Quad (2x2)
cards_2x2.jpg
SBS and Quad images that don't follow this convention will not be detected as a valid file type, and will also fail decoding.
Multiview Operations
We can now perform a several operations on the Multiview Image obtained from the decoder. This includes disparity estimation and View Synthesis. 
You can perform these operations on the Multiview Image using the MultiviewSynthesizer2. You can obtain an instance of this class as shown below.
Kotlin
Java
val synthesizer = MultiviewSynthesizer2.createMultiviewSynthesizer(context)
MultiviewSynthesizer2 synthesizer = MultiviewSynthesizer2.createMultiviewSynthesizer(context);
Disparity Estimation
Images decoded using the Multiview Image Decoder may or may not have depth information, depending on how they were encoded. In order to generate multiple views for a Multiview Image, we need to estimate disparity.
You can populate the disparity maps in a Multiview Image using the MultiviewSynthesizer
Kotlin
Java
val synthesizer = MultiviewSynthesizer2.createMultiviewSynthesizer(context)
synthesizer.populateDisparityMaps(multiviewImage)
MultiviewSynthesizer2 synthesizer = MultiviewSynthesizer2.createMultiviewSynthesizer(context);
synthesizer.populateDisparityMaps(multiviewImage);
View Synthesis
The final steps to display a Multiview image on a Leia Lightfield device is to generate multiple views. This step performs View Synthesis to generate 4 viewpoints and returns a Bitmap representing a Quad.
Kotlin
Java
val synthesizer = MultiviewSynthesizer2.createMultiviewSynthesizer(context)
val quadBitmap = synthesizer.toQuadBitmap(multiviewImage)
MultiviewSynthesizer2 synthesizer = MultiviewSynthesizer2.createMultiviewSynthesizer(context);
Bitmap quadBitmap = synthesizer.toQuadBitmap(multiviewImage);
The Quad bitmap can be used to display the image on the device using the QuadView
Kotlin
Java
quadView.setQuadBitmap(quadBitmap)
quadView.setQuadBitmap(quadBitmap);
​

Display SBS Images in 4V

Play Quad Videos in 4V

Last modified 2yr ago

Multiview File Type	Image File Type
`MultiviewFileType.H4V_BINARY`	Leia Image Format
`MultiviewFileType.H4V_XMP`	Leia Image Format
`MultiviewFileType.TWO_BY_ONE`	SBS (2x1)
`MultiviewFileType.TWO_BY_TWO`	Quad (2x2)

File Type	Valid File name
SBS (2x1)	cards_2x1.jpg
Quad (2x2)	cards_2x2.jpg