ModiFace Is Transforming the Beauty Industry With Augmented Reality

Startup’s platform lets customers try out different looks virtually

25 July 2016

IEEE Senior Member Parham Aarabi had no idea his facial-recognition software, which began as a gag in 2006 when he posted it online, eventually would be adopted by the cosmetics industry. In about a week, more than a million people tried what he called ModiFace, which allowed users to modify their face on their computer. They could appear with a smaller nose, say, bigger lips, or a different eye color. When the server crashed, Aarabi suspected he was on to something.

Today ModiFace is the most widely used augmented reality platform for the beauty industry. In a store, customers use a tablet or a desktop computer to see what they would look like after virtually applying any number of cosmetics. Colors are superimposed realistically on the person’s face. After viewing their image on the screen, people can change what they see—the shade of their lipstick, for example.

Dermatologists and plastic surgeons also use the platform to show patients what they would look like after a face-lift or other treatment.

Aarabi presented ModiFace at the Augmented World Expo, in Santa Clara, Calif. The latest version, which is scheduled to debut this year, allows people to change makeup colors with facial gestures, such as raising the eyebrows for a new eye-shadow shade, instead of clicking a button on the screen.

A PLEASANT SURPRISE

ModiFace had its genesis when Aarabi, a 21-year-old Ph.D. student at Stanford in 1999, set out to develop a speech-recognition platform to read lips. He relied on cameras and microphones to detect the location and movement of the speaker’s mouth. Reading lips could be useful, his thinking went, during video chats in noisy environments or for decoding conversations on security cameras. Although the project never took off, it laid the foundation for what was to come.

Aarabi became an electrical and computer engineering professor at the University of Toronto in 2001. There he developed more advanced facial-recognition software for his original system. He called the system ModiFace. He then uploaded the software to a public website so people could try it out. Media outlets reported on the innovation, and users flocked to it. Soon after, Allergan, the pharmaceutical company that had developed Botox, contacted Aarabi. The company asked if he could customize the platform to simulate Botox treatments on patients.

That’s when Aarabi realized that he had more than a novelty—he had a business opportunity. He spent months consulting with medical experts, among others, to get the simulation as accurate as possible. In 2007 Allergan launched its Juvéderm Visualizer app, which allows a plastic surgeon or dermatologist to upload a photo of a patient and then simulate the effect of antiaging treatments on the image. A click of a button alters the skin’s appearance.

After two years of working with Allergan and other medical and pharmaceutical companies, Aarabi was ready to expand. He turned to the cosmetics industry. The visual effects available through ModiFace made it possible to superimpose makeup on a person’s face on a computer screen as if the person were looking into a mirror. Initially, Aarabi had relied on a still photo of the subject; now he uses a video camera.

“When you put on makeup in the store, you’re likely to try just a handful of cosmetics,” Aarabi says. “With ModiFace, you can try hundreds of cosmetic looks in a matter of minutes before you buy anything.”

L’Oréal, Sephora, Urban Decay, Yves Rocher, and more than 40 other brands use ModiFace to help customers try on cosmetics virtually. A customer looks at the screen. A video camera focuses on the face. The person clicks a button for, say, lipstick and then can make different shades appear seamlessly. It’s also possible to move one’s face from side to side to note how the makeup looks from various angles.

DIFFERENT APPLICATIONS

ModiFace charges each company an annual licensing fee. It depends on how the platform is used—whether on the company’s website, as a mobile app, or in stores, as well as any custom features it might request.

Aarabi’s privately held company, ModiFace Inc., had annual revenue last year of approximately US $10 million, he says, adding that the company has been profitable since its start in 2007.

It’s also a moneymaker for the brands who embrace it. Those that added the platform to their websites saw a first-year jump of as much as 84 percent in online sales and up to 31 percent for in-store purchases, according to Aarabi.

His advice to aspiring entrepreneurs is to “follow the data.” Pay attention to market reports and industry growth areas, he says, warning that good ideas don’t always make for good businesses.

Starting with no experience in the beauty industry, he really had to be convinced when rolling out ModiFace. “I only had data to rely on to help me make my decision about which area to move into,” he says. “And it was the right one.”

Learn More