Music Recommendation System using Nvidia Jetson Nano AI

Music Recommendation System based on User Facial Emotional Expression using Nvidia Jetson Nano AI

Music recommendation detect the user’s face using online web application, identify the facial emotional expression by machine learning algorithms and then recommend a songs playlist based on the detected mood.

A user’s emotion or mood can be detected by his/her facial expressions. These expressions can be derived from the live feed from the system’s camera. A lot of research is being conducted in the field of Computer Vision and Machine Learning (ML), where machines are trained to identify various human emotions or moods.

Human emotions can be broadly classified as: fear, disgust, anger, surprise, sad, happy and neutral. A large number of other emotions such as cheerful which is a variation of happy and contempt can be categorized under this umbrella of emotions. These emotions are very subtle. Facial muscle contortions are very minimal and detecting these differences can be very challenging as even a small difference results in different expressions. Also, expressions of different or even the same people might vary for the same emotion, as emotions are hugely context dependent. While the focus can on only those areas of the face which display a maximum of emotions like around the mouth and eyes, how these gestures are extracted and categorized is still an important question.

Machine learning algorithms have proven to be very useful in pattern recognition and classification, and hence can be used for mood detection as well.

Model uses Deep Neural Network (DNN) with Convolutional Neural Network (CNN) of Keras with multiple layer of convolutional and trained on Nvidia Jetson Nano AI. CNN was used with Keras to train and test model for seven classes — happy, angry, neutral, sad, surprise, fear and disgust. We trained it for 40 epochs and achieved an accuracy of approximately 95%.

Nvidia Jetson Nano AI 2GB Developer Kit.

NVIDIA® Jetson Nano™ Developer Kit is a small, powerful computer that lets you run multiple neural networks in parallel for applications like image classification, object detection, segmentation, and speech processing. All in an easy-to-use platform that runs in as little as 5 watts.

NVIDIA JetPack SDK is the most comprehensive solution for building end-to-end accelerated AI applications. All Jetson modules and developer kits are supported by JetPack SDK. Model use JetPack version 4.6 which is the latest production release and supports all Jetson modules.

Installation of Tensorflow in Nvidia Jetson Nano.

sudo apt-get remove --purge libreoffice*
sudo apt-get remove --purge thunderbird*\

Create swap file

sudo fallocate -l 10.0G /swapfile1
sudo chmod 600 /swapfile1
sudo mkswap /swapfile1~~~~~~~~~~  add line ~~~~~~~~~
sudo vim /etc/fstab
/swapfile1 swap swap defaults 0 0

Cuda Configuration

~~~~~~~~~~  add line ~~~~~~~~~~
vim ~/.bashrc
export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}export LD_LIBRARY_PATh=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_P ATH}}export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libgomp.so.1

Update and Upgrade a System

sudo apt-get update
sudo apt-get upgrade

Install Some Required Packages

sudo apt install curl
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
sudo python3 get-pip.py
sudo apt-get install libopenblas-base libopenmpi-dev~~~~~~~~~~  add line ~~~~~~~~~~
vim ~/.bashrc
export OPENBLAS_CORETYPE=ARMV8source ~/.bashrcsudo pip3 install pillow

Install Torch

curl -LO https://nvidia.box.com/shared/static/p57jwntv436lfrd78inwl7iml6p13fzh.whlmv p57jwntv436lfrd78inwl7iml6p13fzh.whl torch-1.8.0-cp36-cp36m-linux_aarch64.whlsudo pip3 install torch-1.8.0-cp36-cp36m-linux_aarch64.whl# Check Torch, output should be “True”
sudo python3 -c "import torch;
print(torch.cuda.is_available())"

Installation of Torchvision

git clone --branch v0.9.1 https://github.com/pytorch/vision torchvision
cd torchvision/
sudo python3 setup.py install
sudo pip3 install numpy==1.19.4Note:
Comment torch, PyYAML and torchvision in requirement.txtsudo pip3 install --ignore-installed PyYAML>=5.3.
sudo pip3 install -r requirements.txt

Start with Project

Download Dataset from
https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/data?select=icml_face_data.csv

Clone the Repository

git clone https://github.com/YashBorikar/Emotion-Detection_Nvidia-Jetson-AI.git

Go to the project directory

cd Emotion-Detection_Nvidia-Jetson-AI

Install dependencies

pip install -r requirements.txt

Start the server

cd Flask
python emotion_detection.py

Demo Video

Reference

[1] Technology used: https://flask.palletsproiects.com/en/2.0.x/

[2] Library used: https://www.tensorflow.org/api

[3] Library used: https://keras.io/https://github.com/iperov/DeepFaceLab

[4] Library used: https://docs.opencv.org/4.x/d9/df8/tutorial_root.html