Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Android Developers Guide to Machine Learning wi...

Android Developers Guide to Machine Learning with MLkit and TensorFlow

Presented at the Google Developer Agency Day Johannesburg 2018.

Spoke about MLKit, Firebase and Custom TensorFlow models.

Rebecca Franks

June 28, 2018
Tweet

More Decks by Rebecca Franks

Other Decks in Programming

Transcript

  1. Machine learning is an application of Artificial Intelligence in which

    we input a lot of data and let the machines learn “by themselves”
  2. val options = FirebaseVisionBarcodeDetectorOptions.Builder() .setBarcodeFormats(FirebaseVisionBarcode.FORMAT_QR_CODE) .build() val image = FirebaseVisionImage.fromBitmap(bitmap)

    val detector = FirebaseVision.getInstance() .getVisionBarcodeDetector(options) detector.detectInImage(image) .addOnSuccessListener { processedBitmap.postValue(barcodeProcessor.drawBoxes(bitmap, it))
  3. val options = FirebaseVisionBarcodeDetectorOptions.Builder() .setBarcodeFormats(FirebaseVisionBarcode.FORMAT_QR_CODE) .build() val image = FirebaseVisionImage.fromBitmap(bitmap)

    val detector = FirebaseVision.getInstance() .getVisionBarcodeDetector(options) detector.detectInImage(image) .addOnSuccessListener { processedBitmap.postValue(barcodeProcessor.drawBoxes(bitmap, it)) var result = String() it.forEach { result += "VALUE TYPE: ${it.valueType} Raw Value: ${it.rawValue}" textResult.postValue(result)
  4. val image = FirebaseVisionImage.fromBitmap(bitmap) val detector = FirebaseVision.getInstance() .getVisionBarcodeDetector(options) detector.detectInImage(image)

    .addOnSuccessListener { processedBitmap.postValue(barcodeProcessor.drawBoxes(bitmap, it)) var result = String() it.forEach { result += "VALUE TYPE: ${it.valueType} Raw Value: ${it.rawValue}" textResult.postValue(result) } }.addOnFailureListener{ textResult.postValue(it.message) }
  5. private fun doOcrDetection(bitmap: Bitmap){ val detector = FirebaseVision.getInstance() .visionTextDetector val

    firebaseImage = FirebaseVisionImage.fromBitmap(bitmap) detector.detectInImage(firebaseImage) .addOnSuccessListener { processedBitmap.postValue(ocrProcessor.drawBoxes(bitmap, it)) var result = String() it.blocks.forEach { result += " " + it.text textResult.postValue(result)
  6. private fun doOcrDetection(bitmap: Bitmap){ val detector = FirebaseVision.getInstance() .visionTextDetector val

    firebaseImage = FirebaseVisionImage.fromBitmap(bitmap) detector.detectInImage(firebaseImage) .addOnSuccessListener { processedBitmap.postValue(ocrProcessor.drawBoxes(bitmap, it)) var result = String() it.blocks.forEach { result += " " + it.text textResult.postValue(result) } } .addOnFailureListener{ Toast.makeText(/../“Error detecting Text $it”/../) }
  7. private fun doOcrDetection(bitmap: Bitmap){ val detector = FirebaseVision.getInstance() .visionTextDetector val

    firebaseImage = FirebaseVisionImage.fromBitmap(bitmap) detector.detectInImage(firebaseImage) .addOnSuccessListener { processedBitmap.postValue(ocrProcessor.drawBoxes(bitmap, it)) var result = String() it.blocks.forEach { result += " " + it.text textResult.postValue(result) } } .addOnFailureListener{ Toast.makeText(/../“Error detecting Text $it”/../) }
  8. Gather Training Data FFMPEG Folders of Images Retrain with new

    images Optimize for mobile Embed in app Store in Firebase App uses model
  9. Retrain with new images python -m scripts.retrain \ --bottleneck_dir=tf_files/bottlenecks \

    --how_many_training_steps=500 \ --model_dir=tf_files/models/ \ --summaries_dir=tf_files/training_summaries/"${ARCHITECTURE}" \ --output_graph=tf_files/retrained_graph.pb \ --output_labels=tf_files/retrained_labels.txt \ --architecture="${ARCHITECTURE}" \ --image_dir=training_data/south_african_chips
  10. Optimize for mobile bazel-bin/tensorflow/contrib/lite/toco/toco \ --input_file=AgencyDay/retrained_graph.pb \ --output_file=AgencyDay/chips_optimized_graph.tflite \ --input_format=TENSORFLOW_GRAPHDEF

    \ --output_format=TFLITE \ --input_shape=1,${IMAGE_SIZE},${IMAGE_SIZE},3 \ --input_array=input \ --output_array=final_result \ --inference_type=FLOAT \ --input_data_type=FLOAT
  11. You don’t need to be a ML Expert to take

    advantage of ML in your apps!