Harvard Students Develop I-XRAY System, Exposing Privacy Risks Through Facial Recognition and AI Technology

Two Harvard University students have developed a system called I-XRAY that demonstrates how personal information can be extracted from a person’s face using advanced technology. AnhPhu Nguyen and Caine Ardayfio combined Meta’s Ray-Ban smart glasses with facial recognition models, large language models (LLMs), and public databases to showcase the ease with which personal data such as names, addresses, and phone numbers can be obtained in real-time.

In a video demonstration, the two students used I-XRAY to identify classmates, uncover their home addresses, and even the names of relatives. The technology stack they developed leverages smart glasses to livestream video feeds to Instagram, where a computer program analyzes the footage, identifies faces, and runs them through facial recognition platforms such as PimEyes and FaceCheck.id. "In our testing, PimEyes was the most effective and accurate to a wide range of people," the students noted.

Once the facial recognition tools match a face with an online image or URL, I-XRAY utilizes an LLM to scrape relevant personal details, such as the individual’s name. From there, the system uses online tools like FastPeopleSearch to gather further information, including age, address, and phone number. The system was also able to expose partial social security numbers (SSNs) using Cloaked.com, raising further concerns about privacy vulnerabilities in the digital age.

Nguyen and Ardayfio emphasized that their project was designed to raise awareness of privacy issues. "The purpose of building this tool is not for misuse, and we are not releasing it," they stated in a document accompanying the project. They highlighted that the project demonstrates the possibility of "extracting someone’s home address and other personal details from just their face on the street" with the technology available today.

I-XRAY’s use of LLMs is particularly significant, as these models allow for the rapid analysis and synthesis of large amounts of information across multiple sources. "The [I-XRAY] system leverages the ability of LLMs to understand, process, and compile vast amounts of information from diverse sources—inferring relationships between online sources, such as linking a name from one article to another," the developers explained.

Meta's Ray-Ban smart glasses have previously raised privacy concerns due to the potential for people to be recorded in public spaces without their consent. To address these concerns, Meta has implemented a privacy light that activates whenever the glasses are recording. However, some users have reported that the light can be difficult to notice in certain environments, such as bright or crowded spaces. Meta’s privacy policy advises users to inform others about the capture LED to ensure transparency and warns against using the glasses to infringe on privacy rights or engage in harmful activities.

As for the data collected by Meta’s smart glasses, it was reported that some images and videos might be used to improve Meta's AI systems. According to Emil Vazquez, Meta’s policy communications manager, “[I]n locations where multimodal AI is available (currently US and Canada), images and videos shared with Meta AI may be used to improve it per our Privacy Policy.”

Nguyen and Ardayfio's I-XRAY project serves as a reminder of the increasing privacy risks posed by emerging technologies and underscores the importance of public awareness in addressing these challenges.