ASL Classifier for Covid-19 Symptoms

Machine Learning project built with scikit-learn and Google’s TensorFlow libraries to identify Covid-19 symptoms as well as alphabetical letters from American Sign Language (ASL) hand signs and gestures. Achieving ~95% accuracy for both alphabet and gesture validation data.

Why did I pursue this project?

With COVID-19 raging throughout the world, I decided to pursue this project to gain experience in machine learning while satisfying my interest in assistive technology. Having planned this project around April 2020 when we still didn’t know much about COVID-19 and emergency rooms had no capacity, I figured that this could be a helpful way for hospitals to more efficiently help those without the ability to speak without coming into contact with them.

Technologies Used

I created my models on Google’s Colab platform using the Keras, TensorFlow and the scikit learn libraries.

Challenges

Having to teach myself how the different libraries worked was a bit of a learning curve, but I was successfully able to find a lot of tutorials on the TensorFlow website as well as on YouTube.

Machine learning models require a lot of data to train. For the Covid related gestures, I manually screenshotted images of people performing the gestures at key points. I did this many times for many different people and gestures to build some sort of dataset to train my model on. It was by no means enough data, so I looked to image augmentation techniques to make more data from the data I already had. I was successfully able to augment the images I had taken to increase the size of the dataset.

Source