Klasifikasi Ekspresi Wajah Menggunakan Region Selected Facial Landmarks Extraction dan Convolutional Neural Network (CNN)

Nainggolan, Cesilia Natasya and Dr. Eng. Fitra Abdurrachman Bachtiar, S.T., M.Eng. and Dr. Eng. Budi Darma Setiawan, S.Kom., M.Cs. (2024) Klasifikasi Ekspresi Wajah Menggunakan Region Selected Facial Landmarks Extraction dan Convolutional Neural Network (CNN). Sarjana thesis, Universitas Brawijaya.

Abstract

Mengenali dan menginterpretasi ekspresi wajah secara akurat merupakan aspek penting dalam bidang affective computing, terutama dalam meningkatkan interaksi manusia dan komputer. Fokus utama dari penelitian ini adalah pengembangan teknik ekstraksi landmark wajah berbasis region wajah dan penerapan Convolutional Neural Network (CNN) untuk klasifikasi ekspresi wajah. Penelitian ini memanfaatkan teknologi Mediapipe untuk mengintegrasikan perhitungan jarak Euclidean antar landmark wajah dan nilai blendshape, dengan metode pemilihan fitur menggunakan Mutual Information (MI), untuk mencapai keakuratan yang lebih tinggi dalam pengenalan ekspresi wajah. Hasil studi menunjukkan bahwa fitur dua dimensi lebih informatif daripada fitur tiga dimensi, terutama di area mulut, pipi, dan dagu. Model yang dikembangkan, yang memanfaatkan 75% fitur jarak Euclidean 3D dengan skor MI tertinggi ditambah dengan landmark dan blendshape score, berhasil meningkatkan akurasi pengenalan. Proses optimasi hyperparameter, termasuk penentuan learning rate, ukuran batch, ukuran kernel, dan jumlah epoch, meningkatkan kinerja model secara signifikan. Dengan akurasi sebesar 91,95% dan waktu prediksi rata-rata 0,0030 detik, model menunjukkan efektivitas yang tinggi, khususnya dalam mengenali ekspresi wajah seperti contempt, happy, neutral, sad dan surprise. Temuan ini memberikan kontribusi penting dalam pengembangan teknologi pengenalan ekspresi wajah, dengan potensi aplikasi yang luas di berbagai industri. Rekomendasi untuk penelitian mendatang termasuk peningkatan eksplorasi fitur ekspresi wajah, integrasi nilai blendshape, dan pemanfaatan dataset yang lebih beragam.

English Abstract

Accurately recognizing and interpreting facial expressions is a crucial aspect in the field of affective computing, particularly in enhancing human-computer interactions. The primary focus of this research is the development of a facial landmark extraction technique based on facial regions and the application of Convolutional Neural Network (CNN) for facial expression classification. This study utilizes Mediapipe technology to integrate Euclidean distance calculations between facial landmarks and blendshape values, along with feature selection using Mutual Information (MI), to achieve higher accuracy in facial expression recognition. The findings indicate that two-dimensional features are more informative than three-dimensional features, especially in areas such as the mouth, cheeks, and chin. The developed model, utilizing 75% of the 3D Euclidean distance features with the highest MI scores combined with landmark and blendshape scores, successfully improved recognition accuracy. The optimization process of hyperparameters, including determining the learning rate, batch size, kernel size, and number of epochs, significantly enhanced the model's performance. With an accuracy of 91,955% and an average prediction time of 0,0030 seconds, the model demonstrates high effectiveness, particularly in recognizing facial expressions such as contempt, happy, neutral, sad dan surprise. These findings contribute significantly to the development of facial expression recognition technology, with potential applications across various industries. Future research recommendations include further exploration of facial expression features, integration of blendshape values, and utilization of more diverse datasets.

Item Type:	Thesis (Sarjana)
Identification Number:	0524150243
Uncontrolled Keywords:	ekspresi wajah, facial landmark, jarak Euclidean, Mediapipe, feature selection, convolutional neural network
Divisions:	Fakultas Ilmu Komputer > Teknik Informatika
Depositing User:	soegeng sugeng
Date Deposited:	25 Apr 2024 07:42
Last Modified:	25 Apr 2024 07:42
URI:	http://repository.ub.ac.id/id/eprint/217789

Text (DALAM MAS EMBARGO) Cesilia Natasya Nainggolan.pdf Restricted to Registered users only Download (9MB)

Actions (login required)

View Item