Tutorial

Creating a Custom Core ML Model Using Python and Turi Create


In the past few years, the use of machine learning approaches to solve problems and perform complex tasks have been increasing. Machine learning enables us to use big data to perform complex tasks, such as image classification and speech recognition.

Furthermore, Apple recently announced a framework built to simplify the process of integrating machine learning models into macOS, iOS, tvOS and watchOS devices, called Core ML. Apple also provided sample Core ML models to test the framework.

When Core ML was first announced, it was quite a challenge for software developers to create their own custom Core ML models because it required previous experience in machine learning.

However, thanks to GraphLab and Apple, now we have Turi Create, a framework that enables us to easily create Core ML models. Turi Create provides us with essential machine learning algorithms, such as k-nearest neighbor, and advanced deep learning algorithms, such as Residual Networks (ResNet), in order to create our own machine learning models.

In this tutorial, we will demonstrate how to create a custom image classification Core ML model and integrate it into an iOS application. To do so, we will use Python 2.7, Turi Create, Swift 4.0 and Core ML.

Before getting started, we will need the following:

  • A computer with a 64-bit processor (x86_64 architecture)
  • Python, version 2.7: Download here
Note: For now, Turi Create only supports Python 2.7.

Let’s Get Started!

To begin, we will first install Turi Create using the Python package manager, pip. The package manager comes with Python when you install it on your machine.

To install Turi Create, go to Terminal.app and type the following command:

After installing the Python package, we will create a new Python project, by doing the following steps:

  1. Create a folder named MLClassifier
  2. Open Xcode and create a new file
  3. xcode-create-new-file
  4. Select an Empty file from any section
  5. xcode-empty-file
  6. Name the file: classifier.py
  7. Save it to the MLClassifier folder

Creating a Dataset

In this example, we will create a machine learning model that classifies whether an image contains rice or soup. To do so, we will have to create a dataset that contains images of both categories. This data set is used to train our own model. You can download the dataset I am using and save it to the MLClassifier folder. Or you can create a custom dataset by following the next steps:

  1. In the MLClassifier folder, create a new folder named dataset.
  2. Inside the dataset folder, create 2 empty folders named rice and soup.
  3. Inside the rice folder, add at least 100 different images of rice.
  4. Inside the soup folder, add at least 100 different images of soup.

The dataset folder contents should look like this:

classifier-folders

Implementing Turi Create

Turi Create simplifies the development of custom machine learning models. You don’t have to be a machine learning expert to add recommendations, object detection, image classification, image similarity or activity classification to your app.

– the official documentation of Turi Create (https://github.com/apple/turicreate)

In this part, we will implement turicreate into our Python project and label each of the images from the dataset we created in the previous step. Let’s implement and configure the framework by doing the following:

  1. Open the classifier.py file and import turicreate by adding this line:
  2. Next, let’s define the dataset path:
  3. And then add the following line to find and load images from the dataset folder:
  4. Continue to add the following line of code. We define image categories based on its folder path:
  5. > Given an image, the goal of an image classifier is to assign it to one of a pre-determined number of labels.

  6. Lastly, save the new labeled dateset as rice_or_soup.sframe. We will use it to train the model
  7. Preview the new labeled dataset on Turi Create

Training & Exporting the Machine Learning model

It’s time to train and export the machine learning model for production use!

To do so, we will train our machine learning model using SqueezeNet architecture option provided by turicreate.

We will be using the SqueezeNet architecture because it takes less time to train; however, it’s recommeneded to use ResNet-50 for more accurate results. In the following steps, I will demonstrate examples of both architectures:

  1. In the classifier.py file, we first load the previously saved rice_or_soup.sframe file. Insert the following line of code in the file:
  2. Next, create training data using 90% of the dataBuffer object we just created and test data using the remaining 10%.
  3. Continue to insert the following code to create the image classifier using the training data and SqueezeNet architecture:
  4. Alternatively, you can use ResNet-50 for more accurate results:

  5. Next, we will evaluate the test data to determine the model accuracy:
  6. Lastly, insert the following code to save the model and export the image classification model as a CoreML model:

Your classifier.py file should look like this:

python-ml

Now it’s time to run our code! To do so, go to Terminal.app and change to the MLClassifier folder. Then run the classifier.py file like this:

You should see the following message. Just be patient and wait.

building-ml-model

A few minutes later… You will have the CoreML model ready to implement into any iOS, macOS, tvOS, or watchOS application!

Integrating the Core ML model into an iOS app

Now we’re ready to integrate the custom CoreML model we have just created into an iOS app. If you’ve read our introductory tutorial of Core ML before, you should have some ideas about how to integrate the CoreML model into an iOS app. Therefore, I will keep this part short and sweet.

  1. First, create a new iOS project using the Single Application template. Name the project to whatever name you like but make sure you use Swift in this example.
  2. To use the model we just created, drag the trained CoreML model (RiceSoupClassifier.mlmodel file) into the project.
  3. add-coreml-model
  4. Import CoreML framework into our project then add the following line in the ViewController.swift file:
  5. Create the CoreML model object:
  6. Create the user interface of the application:
  7. Add UIImagePickerController protocol calls to the ViewController class:
  8. Create an importFromCameraRoll method:
  9. In the viewDidLoad method, connect the import button to the import method and add the button the subview:
  10. Next, create a UIImage extension and add the following function to convert a UIImage object into CVPixelBuffer:
  11. Analyze the imported image and display the result in a UILabel by adding the following method into the ViewController class:

Ready to Test

That’s it! Have fun. Run the app and click the Import button to test it out. Now you should understand how to create custom machine learning models and implement them into your applications!

coreml-demo

To download the full project, you can find it on GitHub. If you have any questions or thoughts on this tutorial, please leave me comment below and let me know.

Tutorial
A Beginning’s Guide to Lottie: Creating Amazing Animations in iOS Apps
iOS
Adding Animated Effects to iOS App Using UIKit Dynamics
Tutorial
ARKit Tutorial: Working with 2D Image Recognition
  • Abhinandan Bedi

    Hi,
    I have followed all the instructions accordingly but I am getting an error that says, “Fatal Python error: PyThreadState_Get: no current thread
    Abort trap: 6”

    Can anyone explain? https://uploads.disquscdn.com/images/7eed903886cf16634c223945305476b15b7fab018c4c34b005dae54048e91bac.png


  • Steve Richardson

    Hi Ahmed,
    This seems to be the simplest for beginners that ive found, thanks.

    I have used the same code to trying and make it classify 3’s and 8’s using about 100 jpegs for each from the mnist collection.

    It just seems to be showing a few errors:
    https://uploads.disquscdn.com/images/e7e13e5727f3fef3280ff268be2512b7ccd251f8d0d39d7d8401b14065ae68c5.png

    1. Why does it think they are unsupported images when they are all .jpg?
    2. How do I increase the iteration limit?
    3. How do I alter this line “data[“number”] = data[“path”].apply(lambda path: “3” if “3” in path else “8”)” to include multiple options such as other numbers?
    4. Is there a more efficient way seeing as ill only be using simple elements such as numbers and arrows, not pictures?

    Thanks!


  • Meertje

    MeertjeMeertje

    Author Reply

    Wow, what an easy&going tutorial. Thanks, this is exactly what I was looking for. I tried tensor flow and stuff, but the leven was too high. Only thing I noticed is that the image size had to be 227 (maybe because of iPhone X ) and the ResNet worked better than squeeze net.


  • Guru Teja Pulavarty

    Thanks for simple tutorial..
    Is there any way to add multiple folders with images data set in the python script

    data[“foodType”] = data[“path”].apply(lambda path: “Rice” if “rice” in path else “Soup”)

    this is for only two categories. How to write the logic for multiple folders?


    • Sachin

      SachinSachin

      Author Reply

      Did you got the solution to create multiple folder inside dataset?


      • Marc

        MarcMarc

        Author Reply

        Hello, can you share the solution pls? Thanks!


        • rajasekhar tolapu

          Try this…

          data[“foodType”] = data[“path”].apply(lambda path: str(os.path.basename(os.path.dirname(path))))


      • rajasekhar tolapu

        Try this,

        data[“foodType”] = data[“path”].apply(lambda path: str(os.path.basename(os.path.dirname(path))))


    • rajasekhar tolapu

      Try this, it will work.
      data[“foodType”] = data[“path”].apply(lambda path: str(os.path.basename(os.path.dirname(path))))


      • Guru Teja Pulavarty

        Sorry, but I cannot figure it out. Can you explain the logic.
        Thanks.


        • rajasekhar tolapu

          for suppose if your datapath is : username/folder/folder/CAT/cat.png
          find Dir name by using os.path.dirname(path) which results username/folder/folder/CAT
          now find the last dir name(which is ur class) os.path.basename() results CAT.


  • Ray

    RayRay

    Author Reply

    Hello Ahmed,

    Thanks for the informative tutorial! I have a question on how to go about identifying “texts” from an image. Would we need to obtain a hundred images of the letter “a”, “b”, “c”, etc? seems like a lot. Can you point us in the right direction regarding this?


  • Retroplayer

    You are a fucking GOD of iOS A.I.


  • Robert Holzapfel

    Hello Ahmed, thank you very much, I am deeply impressed although beeing a 56 years old man. 🙂
    I have one question: I am very new to machine learning and wherever I read, I find ‘visual recognition’ as examples for machine learning and I really would like to know, whether TuriCreate is also helpful for creating a custom model for e.g. a card play? What do you think?


  • vsplc test

    vsplc testvsplc test

    Author Reply

    i have tried the same datase.but for image different from rice & soup it also gives prediction as rice or soup.


  • iOS Dev

    iOS DeviOS Dev

    Author Reply

    Hello, I am facing following error while running py script.

    File “classifier.py”, line 19, in

    evaluations = model.evaluate(testingBuffers)

    File “/usr/local/lib/python2.7/site-packages/turicreate/toolkits/image_classifier/image_classifier.py”, line 553, in evaluate

    return self.classifier.evaluate(extracted_features, metric = metric)

    File “/usr/local/lib/python2.7/site-packages/turicreate/toolkits/classifier/logistic_classifier.py”, line 839, in evaluate

    metric=metric)

    File “/usr/local/lib/python2.7/site-packages/turicreate/toolkits/_supervised_learning.py”, line 158, in evaluate

    dataset, missing_value_action, metric);

    File “/usr/local/lib/python2.7/site-packages/turicreate/extensions.py”, line 292, in

    ret = lambda *args, **kwargs: self.__run_class_function(name, args, kwargs)

    File “/usr/local/lib/python2.7/site-packages/turicreate/extensions.py”, line 279, in __run_class_function

    raise _ToolkitError(exc)

    Please revert back on this. Thanks in advance.


Shares