JENNY LAY-FLURRIE – Stories from inside Microsoft’s journey to design a more accessible world

by Bill Briggs | Microsoft News May 5, 2020

She did not want to stop at the hospital. She had a flight to catch, an accessibility conference to attend in California, and more people to enlist in her journey to build a more inclusive world.

She had to get to that event.

But Jenny Lay-Flurrie’s left leg was positively throbbing. The mysterious pain, which had erupted 24 hours earlier, was only growing. So, despite the packed bag in her car and the pressing mission on her mind, she grudgingly agreed when her husband, Tom, suggested that they pause their drive to the Seattle airport and instead visit a nearby emergency room.

Tom’s insistence saved her life, she would say later.

Within an hour, doctors had their diagnosis: A 2-foot-long blood clot snaked from her foot to her stomach, reaching perilously close to her lungs. A previously undetected anatomical defect was the cause. That day in March 2019, Lay-Flurrie was admitted to the intensive care unit. Her flight was canceled. Emergency surgery was scheduled. More would follow.

Jenny Lay-Flurrie with her husband Tom McCleery.

“I never expected, at my age, to hear a message from doctors: ‘You may not live through this,’” she recalls.

Surgeons successfully reduced the clot’s size. But the embolism caused long-term leg damage. After leaving the hospital, she relied on two canes to walk. The leg began to heal. More than a year later, she still needed her two canes on some days.

Yet to the surprise of no one in her life, Lay-Flurrie found both whimsy and wisdom in the harrowing episode.

First the humor. She decided to name her clot. She called it “Gerry.” Then she named her canes, a.k.a. her “trusty steeds,” dubbing them “Michael” and “Rosie.” She derided Gerry as “stubborn, sneaky and attention seeking” but soon reported that “Gerry and I have figured out how to cohabitate.”

Next came the insight. Microsoft’s chief accessibility officer – a tech exec who is profoundly deaf – now had a visible disability. More than 1 billion people live with disabilities and about 70% of those are not immediately apparent, such as deafness. On that level, the clot had shoved Lay-Flurrie into a somewhat different reality.

Early in her career, until about age 30, she purposely hid her severe, ever-increasing hearing loss, originally caused by childhood measles and multiple ear infections. Back then, she feared it would define her. So, no hearing aids, no sign language interpreters. Video captioning at work was not yet a thing.

But her perfect diction and exceptional lip-reading skills – honed by practicing in the mirror as a little girl – allowed her to cloak the deafness. Of course, that made workdays exhausting. In time, Lay-Flurrie accepted and then celebrated the disability, though some colleagues still didn’t know she had one.

When Gerry came along, everyone saw the canes.

Strangers asked Lay-Flurrie how she’d been injured. Other people offered advice, some of it helpful. A few well-meaning folks spotted her trudging through hallways and scurried to open doors, once causing Lay-Flurrie to lose her balance and crash to the floor.

Yet all of it gave her a fresh view of how millions of folks – people for whom she advocates – depend on mobility equipment, power entry doors and disabled parking spots – and how they often navigate their days in front of wide eyes filled with fascination, pity or both.

“I’ve had 30, 40 years to get used to the deafness thing,” Lay-Flurrie says. “This thing literally happened in the space of 90 minutes. The learning was immense. Still is. Every time those canes come out, the questions come with them.

“It’s in no way representative of what other people live with on a permanent basis,” she adds. “But good grief, there are things we need to do better. This experience has been a good reminder of why we need people with disabilities to be in the process of product design.”

The episode laid bare many of the personal traits that have aided Lay-Flurrie since becoming Microsoft’s accessibility chief in 2016 – characteristics that enabled her to bring big changes to a big company.

Colleagues say Lay-Flurrie’s leadership style blends relentless honesty, contagious energy and masterful communication. They describe her as unapologetic about who she is – or what must be fixed – charismatic, convincing and empathetic. They say she is utterly grounded yet adept at challenging coworkers to rise to the moment.

The Gerry experience only deepened Lay-Flurrie’s drive to elevate accessibility from afterthought to foresight, from a corporate novelty once beset by ROI questions to simply smart business.

That calling was heard by many across the company, igniting a shared spirit to reshape Microsoft and its products while also influencing change inside industries near and abroad.

It also meshed with a new business culture fostered by Microsoft Chief Executive Officer Satya Nadella – a culture that “needed to be about realizing our personal passions and using Microsoft as a platform to pursue that passion,” as Nadella wrote in his 2017 book “Hit Refresh.”

“For me, my greatest satisfaction has come from my passion to see technology become more accessible for people with disabilities,” he wrote, “and to help improve their lives in a myriad of ways.”

Capturing the complete portrait of Microsoft’s accessibility progress would require far more than a single, lengthy post. That story would fill a fat book.

To see the birth of the Inclusive Tech Lab, you’ll need to trek back to the year 2016, locate Bryce Johnson on the Microsoft campus and take a quick look under his desk.

There, piled into several metal toolboxes, Johnson kept a collection of alternative gaming controllers, all made by different companies. He would often pluck them from the boxes, then place them in meeting rooms or other campus gathering spots, encouraging colleagues to try them out.

“We needed to show people that not everyone can use a traditional controller,” says Johnson, the inclusive lead for Microsoft devices.

In other words, Johnson helped demonstrate how some consumers were not using Xbox – because they were physically unable to manipulate the controls. Other Xbox users, meanwhile, were forced to hack their controllers to match their disabilities.

Across the Xbox team – and throughout the disabled gaming community – the seeds were sewn for a revolution that, in 2018, culminated with the launch of the Xbox Adaptive Controller, designed for gamers with limited mobility.

The Xbox Adaptive Controller became part of a growing list of accessible Microsoft products dreamed up and developed since 2014, many “hacked” on the fly by company employees to boost productivity at work and fun at home.

“Xbox had such a clear mobility problem,” Johnson adds. “We had a barrier that we needed to get rid of.”

In fact, the vision for an Xbox controller that many more people could adapt and use flowed from a team of Xbox employees, says Chris Capossela, Microsoft’s chief marketing officer.

“It’s just another example of teams being innovative in their own spaces, totally independent from top-down management,” Capossela says. “People feel like they can take risks like this, do unusual things. We’ve unleashed this machine.”

That machine geared up as Microsoft President Brad Smith prioritized accessibility across all products, touted that quest in his conversations with other senior leaders at the company and, in 2018, invested millions of dollars toward the mission.

Smith funded the AI for Accessibility program as part of Microsoft’s AI for Good initiative, investing at least $25 million over five years, including grants to organizations and entrepreneurs building artificial intelligence tools for people with disabilities.

“Back in 2016, we doubled down on accessibility and our vision to empower more than 1 billion people around the world with disabilities by providing them with technology,” Smith says.

“What we have learned since is that it takes a village, an army of passionate, incredibly talented people to drive change,” Smith adds. “But when you activate a company like Microsoft, amazing things can happen.”

So, back to those toolboxes that once sat beneath his desk. Johnson and his team realized they needed a dedicated place where people could drop by, interact with the alternative gaming controllers and find the empathy to develop something fresh, something more inclusive.

They needed a spot where people could feel what it’s like to be a gamer with a disability, an embassy to show other Xbox employees “that this was a problem for a certain segment of the population,” Johnson says.

With minimal budget and scrappy attitudes, they took over a room at Microsoft, painted the walls green and gray, installed desks, office chairs, tables and sofas purchased from Ikea, made it all as accessible as possible and, in 2017, launched the Inclusive Tech Lab.

It is an airy space filled with individual nooks furnished with flat screens and gaming consoles where more than 7,000 visitors have sampled a patchwork of gaming demos that force users to compete while using parts of their bodies – a knee or a foot or a mouth. They can control a game with their voice or blow into a sip-and-puff joystick used by gamers with quadriplegia.

In that lab – designed “for people with disabilities, not about people with disabilities” – the Xbox Adaptive Controller came to life, Johnson says. It has large, programmable buttons and connections to external switches, buttons, mounts and joysticks, letting users create a custom controller experience.

“It was the first thing we started but the last thing we launched,” Johnson says.

In early 2019, the Xbox Adaptive Controller was featured in its own Super Bowl ad as young gamers with disabilities focused, played, laughed, cheered and tasted victory. The commercial offered the tagline: “When everybody plays, we all win.”

“This is the thing,” Johnson says. “We, as Microsoft employees, live off this idea of empowerment. We all want to make a difference.

“Accessibility before 2015 didn’t feel like we were making a difference,” he adds. “It felt like we were throwing buckets out of a leaky boat. The new focus on accessibility brought … the fun and the passion. And honestly, the community.”

An array of other accessible Microsoft inventions and inventors has broadened both the product catalogue – and that community.

At his home in northern Virginia, Eric Bridges discovered some creative uses for a Microsoft app called Seeing AI.

Bridges, who is blind and serves as executive director for the American Council of the Blind, activated Seeing AI on his smart phone to scan schoolwork completed by his son, Tyler, 5. The app, which describes the world around you, read Tyler’s hand-written answers, allowing Bridges to review the tasks and guide his son.

There was more.

“I also use it for wine. We’re in a wine club and we just got our shipment. So, I pulled out Seeing AI to read the wine labels,” he says.

“The cool thing is, there’s a suite of Microsoft apps out there that blind people can use, including Seeing AI and Microsoft Soundscape – which gives you information about where you are and what’s around you. Microsoft developed these applications and makes them free to use,” Bridges adds.

Seeing AI, which uses computer vision, speaks text as soon as it appears in front of the camera. It also scans barcodes to identify products, describes perceived colors, identifies currency bills, and recognizes friends and the people nearby, including their emotions.

Bridges worked with the creators of Seeing AI on the initial beta testing. The driving force behind the app, Microsoft software engineering manager Saqib Shaikh, has something else in common with Bridges. Shaikh is blind. He lost his sight at age 7. During his college days in the U.K., Shaikh began noodling with the concept of tech that can see.

“We had ideation sessions in the dormitory,” Shaikh has said. “We’d say things like, ‘Okay, we should make a pair of glasses with a camera on it that can look around at everything and describe it out loud.”

As an engineer, Shaikh collaborated with scientists at Microsoft Research, the company’s research and development arm. They relied on cloud computing, which enables a computer to describe the scene in a photo – a breakthrough for Shaikh’s dream. His team launched Seeing AI in 2017.

“Seeing AI was successful, in part, because so many employees at Microsoft who are blind and low vision were a part of it,” says Mary Bellard, Microsoft’s senior architect lead for accessibility. In 2015, she played an early role in helping to develop the app at the company’s annual Global Hackathon.

“People with disabilities need to be more involved beyond user research. It is not good enough to just say, ‘Here, we made a product. Is this useful?’ It has to be more ingrained than that,” Bellard adds.

Globally, only one in 10 people with disabilities has access to assistive technologies and products, according to the World Health Organization. AI solutions may offer a path to close that gap.

That helped lead Microsoft to launch its AI for Accessibility initiative. Bellard manages the program.

Grants are already supporting innovations like Braille AI Tutor, which helps students practice and improve their Braille literacy skills via gamification.

It’s built on the very same spirit that fueled Seeing AI, Bellard says.

“When the history of accessibility is written,” Bellard says, “there’s going to be a big chapter on Apple’s iPhone and, at some point in that book, there’s also going to be a chapter on Seeing AI.”

Years later, Rico Malvar can recall the conversation. And the words still make him smile.

Steve Gleason, a retired NFL safety whose blocked punt electrified a New Orleans Superdome crowd in the first game after Hurricane Katrina, and a man whose body was later stilled by amyotrophic lateral sclerosis (ALS), once revealed a fresh hope to Lay-Flurrie.

“Look,” Gleason said, his voice softening though still audible in 2014, “I’m here in a wheelchair. I can’t move anything. I just have my eyes.

“Can you give me some eye tracking,” he asked, “so I can play with my son and argue with my wife?”

That conversation had started with an email Gleason sent to Microsoft after he was featured in the company’s 2014 Super Bowl ad on technology’s power to help humanity achieve a greater good. His email reached Microsoft CEO Nadella, who shared it with Lay-Flurrie.

The challenge galvanized a group of Microsoft engineers, program managers, marketers and advocates – including Lay-Flurrie – to design and build a wheelchair Gleason could drive with his eyes.

They dubbed themselves the Ability Eye Gaze team and formed to compete at the 2014 Microsoft Global Hackathon. It marked Lay-Flurrie’s first Ability Hack – then a new category within the Global Hackathon, enabling employees to focus on creating technologies that enrich the lives of people with disabilities.

Lay-Flurrie had 10 Ability Hack projects that year. She kicked off Ability Eye Gaze after speaking with Gleason, his father-in-law and one of the leaders of Team Gleason, a foundation started in 2011 by Gleason, his friends and family. The organization helps provide technology, equipment and services to people with neuromuscular diseases or injuries. Lay-Flurrie later became a foundation board member.

The Ability Eye Gaze team, which spanned more than 30 members, put Gleason’s idea into action. They overhauled a wheelchair, strapped it with electronic gadgetry as well as slices of Styrofoam and lots of duct tape. Gleason was able to operate the chair just with his eyes.

That year, the Eye Gaze Wheelchair won the Hackathon’s grand prize.

From the project’s inception to its completion, Gleason remained their guiding star and their honest chief analyst. Later that year, when Nadella met the Ability Eye Gaze team to celebrate the win, Malvar was introduced to Gleason and soon began heading a team to bring Eye Gaze to the market.

“Steve is one of the most wonderful people you could possibly meet,” says Malvar, leader of the Microsoft Research NExT Enable group, which creates technology innovations that improve the lives of people with disabilities, including Microsoft Soundscape and the Hands-Free Keyboard.

“He’s very critical, very pragmatic. If we bring him prototypes, he will say, ‘Improve it this way’ or ‘Let’s do this.’ It’s wonderful. We learn a ton from him.”

Gleason later described how the experience changed his life.

“I was at a point in my disease progression where I could no longer drive myself. Stuck,” Gleason said.

“I am no longer confined to my wheelchair; I am set free by my wheelchair.”

Eventually, that Hackathon invention resulted in a new accessibility feature called Eye Control for Windows 10, which allows users to operate an onscreen mouse, keyboard and text-to-speech features, using only their eyes.