A project by Emu Unicorn Sauce
The objective of the workshop is to showcase the use of AI hand tracking through learning sign language.
In this workshop, participants will learn the American Sign Language(ASL) alphabet through the use of AI that uses hand tracking through a computer camera. By selecting the wanted letter, students will learn and practice ASL through the website.
- Difficulty Level: Beginner/Intermediate
- Target Audience: Middle School - College
- Duration of Workshop: between 30 min - 45 mins
- Needed Materials: Computer with camera
- Learning Outcomes: The primary goal of this project is to teach participants about how AI tracking is being used
- Your Main Technology AI hand tracking, it works by using computer vision to analyze and detect an object's position
- Technology Ambassador Program (TAP) is a project-based class that provides a collaborative environment for students to work with their fellow classmates on a semester-long project using technologies of their choice. TAP strives to increase participation in IT through numerous outreach activities and workshops that are designed to showcase the creative and fun side of technology.
(From left to right: Keren Zhang, Joey Chen, Ivana Hernandez.)
- Joey Chen
- Ivana Hernandez
- Keren Zhang
- Dr. Wei Jin
- Dr. Xin Xu
Participants begin by exploring an AI-powered hand tracking website that provides real-time feedback on their gestures. Unlike traditional sign language learning through videos, where learners often struggle to know whether their hand positions are accurate, this system uses computer vision to mirror participants’ movements and measure their accuracy.
During the workshop, students will follow guided exercises in which an AI model recognizes their hand shapes and displays hand landmarks over their hand. The system highlights the percentage of accuracy (e.g., 90%, 98%, 100%) to help participants understand their progress before moving to the next gesture.
The project leverages technologies such as MediaPipe for hand tracking, allowing users to see landmarks overlay their hand. By doing so, participants not only practice sign language but also gain an interactive and immersive learning experience.
This approach to learning sign language is more flexible and accessible. Learners can study anywhere, anytime, while receiving immediate feedback, which significantly increases engagement and retention. This interactive model transforms sign language learning into a process that is visual, participatory, and highly effective.
In addition to practicing sign language, participants will also be introduced to the underlying technology behind the system. The workshop will cover how AI hand tracking works, including concepts of computer vision, gesture recognition, and how different components (such as MediaPipe and accuracy scoring) are integrated into one interactive platform. By walking through these technical elements, participants will not only experience the application but also gain insight into how to build such a system themselves.
- Ivana Hernandez, Joey Chen, Keren Zhang, Dr. Wei Jin, Dr. Xin Xu. "Teaching Sign Language Using AI Hand Tracking Technology", Consortium for Computing Sciences in Colleges: Southeastern Region (CCSC:SE), November 7-8, 2025, Macon, GA
- Ivana Hernandez, Joey Chen, Keren Zhang, Dr. Wei Jin, Dr. Xin Xu. "Teaching Sign Language Using AI Hand Tracking Technology", CREATE Conference, November 21, 2025, Georgia Gwinnett College
- Super Saturday Series (S3), November 15, 2025, GGC – Promote IT and STEM career to middle school girls and boys
- Tap Expo, October 9, 2025, GGC – Promote IT field and the TAP program to college students
- Class Workshops, November 4-14, 2025, GGC - Promote IT to non-IT college students
- Cross Keys High School Workshop, November 14, 2025, GGC - Promote IT to high school students at Cross Keys High School
If you're interested in more workshops that utilize AI, check it out AI Diva!
- MediaPipe is an open-source, cross-platform machine-learning framework created by Google for building real-time perception pipelines. It provides ready-made solutions for tasks like hand tracking, face detection, pose estimation, and object tracking, using highly optimized ML models. MediaPipe works on mobile, web, desktop, and embedded systems, making it popular for computer-vision and augmented-reality applications.
- PyCharm is a professional integrated development environment (IDE) specifically designed for Python development. It provides powerful tools like intelligent code completion, debugging, refactoring, and integrated testing support.
- OnRender is a cloud hosting platform that lets developers deploy web apps, static sites, background workers, and databases with minimal configuration. It automates builds, scaling, SSL, and deployment pipelines, making it popular for hosting full-stack applications. Render offers a simple UI and Git-based deployments.
- Vite is a fast front-end development build tool created by Evan You, designed to provide near-instant server startup and lightning-fast hot module replacement (HMR). It uses native ES modules in development and Rollup for optimized production builds. Vite is widely used in modern frameworks like Vue, React, and Svelte for its speed and simplicity.
- Search up a the ASL alphabet and use the pictures with Gesture Control
- Go to our website
- In the dropdown menu select 'Webcam'.
- Click 'Start Camera' and allow camera if the option appears.
- If the status says 'Connected' Select a letter you like to sign.
- Click 'Start Recognition'.
- Sign the letter you picked
- When you're done click 'Stop' next to 'Start Recognition'
This is the demo video for our gesture-based web app project!
Watch on YouTube: https://youtu.be/G03tb9w6HDg