Machine Learning made Easy :- Image Segmentation using Huawei ML Kit and Kotlin etc. - Huawei Developers

{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Introduction
Image Segmentation – This service can extract segments from image. In this service static image form camera and gallery both are supported and it also supports dynamic camera streams.
Internally it uses Mean Intersection-Over-Union which is a common evaluation metric for semantic image segmentation.
Once segmentation is done it can create coordinate array of each element like human, plant animal etc. Once you get this array you can replace any value with other one to create some wonderful results.
Here we will learn on how we can extract a user from an image, change his background and create a new wonderful result using Huawei ML Kit.
I will divide this process in 4 steps
1. How to choose an image from gallery or camera
2. Once we get the result how to send it to Huawei ML Kit
3. How to complete the processing of image with desired result
4. How to showcase newly extracted image onto the screen
Have you noticed how easy it is to select an image and change its background as per our choice.
One more thing to notice here is how sharp the selection was and which results to how clear the extracted image is looking.
Once we get the segmented image we can create different results as well.
So let’s get to the point
Step 1:
To choose an image from gallery or camera use the below code.
By Camera:
Code:
private fun uploadByCamera() {
val takePicture = Intent(MediaStore.ACTION_IMAGE_CAPTURE)
startActivityForResult(takePicture, 1222)
}
By Gallery:
Code:
private fun uploadByGallery() {
val photoPickerIntent = Intent(Intent.ACTION_PICK)
photoPickerIntent.type = "image/*"
photoPickerIntent.putExtra(Intent.EXTRA_LOCAL_ONLY, false)
startActivityForResult(
Intent.createChooser(photoPickerIntent, "Choosing picture from gallery"),
loadImageGalleryCode
)
}
Step 2:
Once the intent is triggered you will be directed to system activities and you have to choose an image.
The control once again comes back to our ImageSegmentActivity in onActivityResult method.
We have to get the bitmap and save it for future use. Below is the code for reference.
Code:
override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
super.onActivityResult(requestCode, resultCode, data)
if (requestCode == loadImageGalleryCode && resultCode == Activity.RESULT_OK && data != null) {
val pickedImage: Uri? = data.data
val filePath = arrayOf(MediaStore.Images.Media.DATA)
val cursor: Cursor? = contentResolver.query(pickedImage!!, filePath, null, null, null)
cursor!!.moveToFirst()
val imagePath: String = cursor.getString(cursor.getColumnIndex(filePath[0]))
val options = BitmapFactory.Options()
options.inPreferredConfig = Bitmap.Config.ARGB_8888
val bitmap: Bitmap = BitmapFactory.decodeFile(imagePath, options)
imageSegmentationViewModel.bitmap.value = bitmap
imageSegmentationViewModel.imageSegmentation()
cursor.close()
} else if(requestCode == loadImageCameraCode && resultCode == Activity.RESULT_OK && data != null){
val bitmap: Bitmap? = data.extras!!["data"] as Bitmap?
imageSegmentationViewModel.bitmap.value = bitmap
imageSegmentationViewModel.imageSegmentation()
}
}
Step 3:
After getting the bitmap I am setting a MutableLiveData which is present in my ImageSegmentationViewModel and calling imageSegmentation() method.
Code:
fun imageSegmentation(){
val setting = MLImageSegmentationSetting.Factory()
.setExact(false)
.setAnalyzerType(MLImageSegmentationSetting.BODY_SEG)
.setScene(MLImageSegmentationScene.ALL)
.create()
analyzer = MLAnalyzerFactory.getInstance()
.getImageSegmentationAnalyzer(setting)
val frame = MLFrame.fromBitmap(bitmap.value)
val task = analyzer.asyncAnalyseFrame(frame)
task.addOnSuccessListener {
bitmap.value = it.foreground
}.addOnFailureListener {
Log.d("Image Segmentation: ", "Error occurred: "+it.message)
}
}
Let us discus about above used line of code in detail
Before we create MLImageSegmentationAnalyzer and as we need to tune our settings to get a desired result, Hence below settings are been changed in order to extract fast human body segment from the image.
Once this MLImageSegmentationSetting object is created for tuning we will pass this to MLImageSegmentationAnalyzer.
setExact(False) : True is whether you want fine segmentation and False is fast segmentation.
setAnalyzerType(MLImageSegmentationSetting.BODY_SEG) : Setting the mode to identify and extract body from image.
setScene(MLImageSegmentationScene.ALL) : On a whole we can extract 4 scenes from image which are mentioned below with their key as well.
MLImageSegmentationScene.ALL: All segmentation results are returned (pixel-level label information, human body image with a transparent background, and gray-scale image with a white human body and black background).
MLImageSegmentationScene.MASK_ONLY: Only pixel-level label information is returned.
MLImageSegmentationScene.FOREGROUND_ONLY: Only a human body image with a transparent background is returned.
MLImageSegmentationScene.GRAYSCALE_ONLY: Only a gray-scale image with a white human body and black background is returned.
For our example we are using MLImageSegmentationScene.ALL as we want all the scenes from the imag
Finally above created MLImageSegmentationSetting object we will be providing to MLImageSegmentationAnalyzer
MLImageSegmentationAnalyzer analyzer = MLAnalyzerFactory.getInstance().getImageSegmentationAnalyzer(setting)
We can also create MLImageSegmentationAnalyzer object by calling below code as well.
MLImageSegmentationAnalyzer analyzer = MLAnalyzerFactory.getInstance().getImageSegmentationAnalyzer();
But in this code customization is very limited.
Next step is to create an MLFrame by below code and ptovide your previously fetched image in bitmap format.
MLFrame frame = MLFrame.fromBitmap(bitmap);
On analyser object we will be calling asyncAnalyseFrame(frame) and providing MLFrame which we recently created.
This will yield you a Task<MLImageSegmentation> object, on this object you will get 2 callbacks.
onSuccess
onFailure
You can save the new resource from onSuccess() and stop the analyzer to release detection resources by analyzer.stop() method.
Step 4:
Here is how we can set bitmap to imageview
Inside our main layout view model was added with name as dashboardViewModel
We also added a custom tag named as customImageSrc
Code:
<ImageView
android:id="@+id/imageView"
customImageSrc="@{dashboardViewModel.bitmap}"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:contentDescription="@string/user_image"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintHorizontal_bias="0.5"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent" />
We created an Adapter class named CustomImageAdapter where we bind our ImageView by @BindingAdapter("customImageSrc") tag.
Once our Mutable livedata gets updated in ViewModel by success callback from Task<MLImageSegmentation>
This imageview will automatically get updated by BindingAdapter.
Code:
class CustomImageAdapter {
companion object{
@JvmStatic
[user=8928444]@Binding[/user]Adapter("customImageSrc")
fun setImageViewResource(imageView: ImageView, bitmap: Bitmap?) {
bitmap?.apply { imageView.setImageBitmap(bitmap) }
}
}
}
I hope you liked this article. I would love to hear your ideas on how you can use this kit in your Applications.
In case you dont have a real device you can check out this article
For more information, you can visit https://forums.developer.huawei.com/forumPortal/en/home