9 API of Firebase Machine Learning

On 3 June 2020, they started offering ML Kit's on-device APIs through a new standalone SDK. Cloud APIs and custom model deployment will continue to be available via Firebase Machine Learning.

Firebase Machine API

Google's machine learning expertise is brought to Android and iOS apps with Firebase Machine Learning, a powerful yet simple-to-use mobile SDK. Whether you're new to machine learning or a seasoned pro, you can get the functionality you need with only a few lines of code.

To get started, you don't need a lot of experience with neural networks or model optimization. Firebase ML, on the other hand, provides handy APIs that allow you to use your custom TensorFlow Lite models in your mobile apps if you are an experienced ML developer.

Check out ML Kit if you're seeking for pre-trained models that run on the device. ML Kit is available for iOS and Android and includes APIs for a variety of applications:

  • Text recognition
  • Image labeling
  • Object detection and tracking
  • Face detection and contour tracing
  • Barcode scanning
  • Language identification
  • Translation
  • Smart Reply
  • Landmark recognition

Text recognition

You can recognize text in over 100 different languages and scripts with Cloud Vision's text recognition API.

You can use this Cloud-based API to automate data entry and extract text from document images, which can be used to improve accessibility or translate documents.

Key Capabilities
  • High-accuracy text recognition - Google Cloud's industry-leading image understanding capabilities powers Firebase ML's text recognition APIs.
  • Appropriate for photographs and documents - APIs that recognise both sparse text in images (such as photos of road signs or business cards) and densely-spaced text in photos of documents.

Image labelling

You may recognise entities in an image using Cloud Vision's image labelling APIs without needing to provide any additional contextual metadata.

Image labelling provides information about the content of photos. When you use the API, you'll get a list of the entities that were identified, such as persons, things, places, and actions. Each label discovered gets a score that represents the ML model's confidence in its relevance. You can use this data to do tasks like automatic metadata generation and content moderation.

Key capabilities
  • Google Cloud's industry-leading image comprehension capacity powers Firebase ML's image labelling API, which can identify photos with 10,000+ labels in a variety of categories.
  • In addition to the label's text description, Firebase ML also returns the label's Google Knowledge Graph entity ID. This ID is the same ID used by the Knowledge Graph Search API for uniquely identifying the entity represented by the label.

Face detection

You can detect faces in an image, identify important facial traits, and get the outlines of discovered faces using ML Kit's face detection API. It's important to note that the API only detects faces and does not recognize people.

Face identification provides the data needed to conduct tasks such as embellishing selfies and portraits, as well as producing avatars from a user's photo. You may use ML Kit in applications like video chat or games that respond to the player's expressions because it can recognize faces in real time.

Key capabilities
  • Recognize and locate the features of the face Get the coordinates of every face detected's eyes, ears, cheeks, nose, and mouth.
  • Recognize the contours of your face. Get the outlines of the faces you've detected, including their eyes, brows, lips, and nose.
  • Recognize different types of facial expressions Determine whether a person is smiling or not by looking at their eyes.

Object detection and tracking

You can recognize and track things in an image or live video feed using ML Kit's on-device Object Detection and Tracking API.

You have the option of classifying observed objects using the API's built-in coarse classifier or your own custom picture classification model. For additional details, see Using a custom TensorFlow Lite model.

It works effectively as the front end of the visual search pipeline because object recognition and tracking takes place on the device.

Key capabilities
  • Object recognition and tracking that is quick Identify things in the image and determine their locations. Objects are tracked across subsequent picture frames.
  • Model for on-device optimization The object identification and tracking model has been designed for mobile devices and is intended for usage in real-time applications on even the most basic devices.

Barcode scanning

You may read data encoded in most standard barcode formats using ML Kit's barcode scanning API. The scanning of barcodes takes place on the device itself, and there is no need for a network connection.

Barcodes are a quick and easy way to get data from the real world into your app. Structured data, such as contact information or WiFi network credentials, can be encoded using 2D formats such as QR codes. Because ML Kit recognises and parses this data automatically, your app can react intelligently when a user scans a barcode.

Key capabilities
  • Automatic format detection - Scan for all supported barcode formats at once without having to specify which one you want, or speed up scanning by restricting the detector to only the formats you want.
  • Works with any orientation - Regardless of their orientation: right-side-up, upside-down, or sideways, barcodes are identified and read.

Language ID

You may determine the language of a string of text using ML Kit's on-device language identification API.

When working with user-provided text, which frequently lacks linguistic information, language identification might be useful.

On-device translation

You can dynamically translate text across more than 50 languages with ML Kit's on-device Translation API.

Key capabilities
  • Support for a wide range of languages You can translate between more than 50 languages. See the entire list.
  • Translation models that have been proven to work Offline mode in the Google Translate app is powered by the same models.
  • Model management that is dynamic By dynamically downloading and managing language packs, you can keep your on-device storage requirements low.

Smart Reply

You may automatically produce relevant replies to messages using ML Kit's Smart Reply API. Smart Reply makes it easier for your users to respond to messages quickly and on devices with restricted input capabilities.

Landmark recognition

You can recognize well-known landmarks in a picture using Cloud Vision's landmark identification API.

When you give a picture to this API, you'll obtain a list of the landmarks that were detected in it, as well as their geographic coordinates and the region of the image where they were discovered. You can utilise this data to produce picture metadata automatically, personalize user experiences based on the material they contribute, and more.

Post a Comment

Previous Post Next Post