An electrical engineering graduate at the University of Brasília (UnB), with a master’s degree at the Federal University of Rio de Janeiro (UFRJ), and a PhD at the Massachusetts Institute of Technology (MIT), Henrique (or Rico) Malvar, from Rio de Janeiro, age 65, completed 25 years at Microsoft in August. During this time, he had a rapid rise and a very successful career. Seven years after being hired, he became the director of the main research laboratory, Microsoft Research, located in Redmond, in the outskirts of Seattle, in the state of Washington, USA, where the company’s headquarters are located.
Soon after, he was invited to be the head scientist of the unit, whose team is made up of over a thousand researchers including computer scientists, engineers, physicists, mathematicians, and other specialists. For 10 years, until 2020, he designed strategies for innovation and helped coordinate the work of the company’s 12 laboratories spread across the globe. During this time, he was recognized by Microsoft with the title of distinguished engineer for the important technological contributions he gave Bill Gates’s company.
Author of 120 patents in the USA and 180 technical articles, Malvar currently leads a team that works on new human-machine interfaces, including accessibility technology, at Microsoft Research. “We work to empower people with disabilities so they can use computers and enter the digital world,” he told Pesquisa FAPESP. His team has already created and launched tools that enable victims of paralysis to use computers using eye movement and for the visually impaired to walk through the city receiving three-dimensional voice guidance from a smartphone.
In this interview, given from his home by video conference, he talks about his career in one the largest information technology companies in the world and his plans for the future.
How did you join Microsoft?
That was a long time ago, in 1997. I never imagined I would have the same job for so many years. But, thinking about it, it is as if I have worked in three or four different companies, because Microsoft has reinvented itself over the years. Today, it is no longer just a software manufacturer, but a company of cloud computer systems, artificial intelligence [AI], and also applications. In the 1980s, after my PhD at MIT, I returned to Brazil and became a professor at UnB, leading the Digital Processing of Signs Group, but I always had the desire to return to the USA. During my PhD, some colleagues from MIT created one of the first commercial videoconferencing companies, called PicTel. And I helped them. When the Gulf crisis hit [conflict in the Middle East at the beginning of the 1990s, resulting in the invasion of Iraq by the USA], nobody wanted to travel, out of fear of terrorist attacks, and the PicTel business, based on the videoconferencing platform, grew a lot. I was in Brazil, when I received a phone call from Brian Hinman, one of the founding partners of PicTel, inviting me to be the company’s research director. My wife and I discussed the pros and cons for a month, until I decided to accept the offer. Everything was going really well at PicTel, when I noticed that Microsoft was doing interesting things in the area of voice and media, and Microsoft Research, its research laboratory, was growing. I decided to send a slightly crazy email to the company CTO [Chief Technology Officer]. In it, I said more or less the following: “The research lab is growing, it has several areas, but you don’t have a multimedia team. And you need this team, because all media is going to turn digital. This is my area; I understand it and can create and lead a team. What do you think?” The following day, to my surprise, I received an email from the Microsoft recruitment sector scheduling an interview. They liked me, made me an offer, and soon after we moved to Seattle.
How was your trajectory in the company?
I started leading a team in my technical area, which is signal processing and multimedia, in other words how the computer and the computing devices work with image, video, and sound signals. At this moment, for example, the program that enables our communication via videoconferencing is using some parts of computer code that I wrote many years ago. After seven years leading the signals processing team, I was invited to be the director of Microsoft Research, in Redmond. It was a leadership and administrative role and I was responsible for the budget, I managed people, I made decisions, and an important resolution was to invest more in the architecture of computers and processors at a time when Microsoft only made software. In 2006, I became a distinguished engineer, a recognition that Microsoft gives to some engineers that have made important contributions to systems architecture changes. Four years later, I was invited to be the head scientist of Microsoft Research, a more strategic guidance role. I took care of collaborative projects involving the company’s 12 global research laboratories. I went a lot to China, to India, to England. About two years ago, I moved again and had the opportunity to create a team focused on accessibility.
When I led Microsoft Research, I decided to invest more in computer architecture at a time when the company only made software
What is the objective of your new team?
To help empower people, especially those with disabilities. To do so, we turn to an area of computing known as human-machine interaction. What is it? How can a blind person, for example, who is not able to see the text on a screen, use a computer or smartphone? And someone with paralysis that cannot move their hands? My work is to create alternatives or improved interfaces so those with disabilities can use computers and take part in the digital world. Our laboratory is split into three areas: Enable, which develops practical accessibility systems; Epic, which is dedicated to virtual and augmented reality; and Ability, focused on research into accessibility.
Has this work already had practical results?
Yes. One of the first products was launched on the Windows 10 operating system. It is called Eye Control. A paralyzed person, with ALS [amyotrophic lateral sclerosis, a degenerative neurological disease that leads to paralysis of bodily movements], or another type of disease, can use it to control the computer with their eyes. In reality, they control the cursor with their eyes. The cursor goes wherever they look. And then there are several interesting things. For example, we cannot show the cursor on the screen, because if it were visible, the psychology of the brain makes you want to follow the cursor, causing confusion for the user: “Am I controlling the cursor, or am I following the cursor?” By the way a person looks at the screen, they can read and see things; staring at a point for more than half a second is the same as a click. This is working really well. It just requires a special infrared camera to be installed, to detect the eye movements. Windows receives the information and the person can control the computer.
Any other innovation?
Our team also created Soundscape, an iPhone application that helps the visually impaired to explore the world. Instead of showing a map on the screen—like conventional applications—it uses a new resource, three-dimensional audio, to guide blind people. The interface with the person is no longer the screen, but the audio instead. With Soundscape, we were able to manipulate the wave propagation so that the sound appears to be coming from a specific point of the environment. The user just has to point the cellphone forwards and the program will inform, for example, that there is a new Chinese restaurant there—and the sound comes from where the restaurant is. Soundscape enables people to walk through the city without needing to ask anybody for help or use a map. Accessibility is something important, because you help more people enter the digital world. We want to remove barriers for those with disabilities. Every time we develop new technologies, the trend is to increase the barriers. New technology demands something different, and for those with disabilities, that new thing is often a hindrance rather than a help.
What is it like working at Microsoft?
It is a cool company that has never been afraid to take risks and push the limits of technology. That is since the period in which it was run by Bill Gates [he is currently not involved with the day-to-day running of the company]. He always really liked the work of Microsoft Research and I had the pleasure of having many discussions with him. I remember one in which he told me that I had to “do it this way.” I replied: “No, I have to do it this other way.” It’s not easy convincing Bill Gates, but that day I managed it. The fact is that, 90% of the time, when he said “you have to do it this way,” he was right. He’s an exceptional person.
Do you have any Brazilian researchers in your accessibility team?
Not in my team, but we have some in Microsoft Research. One of them is Karin Strauss, who leads cutting-edge research in the area of information storage in DNA [in which the chips would be made of synthetic DNA fibers]. Imagine if we could use DNA as a means of storing information, by using bioengineering tools. The storage density is immense. When it is working, we will be able to take everything that is on the internet, all the videos, images, texts, hundreds of petabytes of information and store them on a disk the size of two shoe boxes. This changes the whole concept of computing, because we wouldn’t need to delete information. It would be extremely important for universities, museums, medical, scientific, and climatology databases. It is a collaborative study with the University of Washington.
If the efficiency of positive results was much higher than 10%, you are probably not taking many risks
Do you have any interaction with research groups in Brazil?
Yes, we have a small team of researchers at Microsoft Brazil—which I myself created some years ago—that works with artificial intelligence and computer vision. They are no longer linked to me, but they develop cool projects. One of them was done with the Hospital 9 de Julho, in São Paulo. It was a pilot project, with good results, for monitoring hospitalized patients using cameras to avoid bed falls. There are lots of patients that think they are well, who hurt themselves trying to get out of bed. Based on artificial intelligence and computer vision in real time, our system warns the nursing staff, by messages, that the patient from room so and so is moving and appears to want to get out of bed. With this, the nurse goes to them before they hurt themselves. We also have a good relationship with the state universities in São Paulo, such as UNICAMP [University of Campinas], USP [University of São Paulo], and the federal universities from Rio de Janeiro and Minas Gerais [UFRJ, UFMG], among others. Some years back we created a program together with FAPESP, the FAPESP Microsoft Research Institute of Information Technology Research. We invest in research projects in Brazil, especially in the environmental field, without restriction. Each partner provided half of the funding [see Pesquisa FAPESP issue nº 135].
You have filed over 120 patents in the USA, either individually or as a co-inventor. Which of them resulted in the greatest benefit for the users?
It’s hard to say. I did a lot for Windows Media [a Microsoft video, audio, and image application], some for Windows and for Office. In 2007, my team created Microsoft Round Table, an innovative 360-degree videoconferencing system. But, from a user-impact point of view, the patents I did related to H264 video compression standards were probably the most important. The H264 format is what we are currently using to see each other in this video conference. During a video conference, it is not possible to take all of the pixels that are in the image and transmit them; it is necessary to compress them first. And that is what H264 does. It is responsible for over 75% of all audio, image, and video traffic on the internet. The Teams, Zoom, YouTube, and Netflix applications use this standard, which was a high-impact technology.
You have already stated that 90% of the research done at Microsoft will come to nothing, but 10% of it will have amazing results. Is the success rate not low?
No, I think that it is a good rate. If the efficiency of positive results was much higher than 10%, then you are probably not taking many risks. And, if it were lower, say 1%, it is because you are not doing a very good job. So, between 5% and 10% of efficiency is a good result. We have some high-risk projects that will take many years to give results. One is in the area of quantum computing. Although commercial models of a quantum computer do not yet exist, we have made important advances on the software side. You might ask: “If the hardware doesn’t exist yet, why are we developing software?” The answer is simple: because the day the computer is ready it is going to demand different algorithms. For the time being, we can emulate the quantum computer in the cloud using the software and test these algorithms. When the quantum computer is working, I will transfer them to it. The person leading this project is a woman, the researcher Krysta Svore, also a distinguished engineer at Microsoft. We really value diversity in the company.
Is it an investment for which the results will still take time to reach society?
Yes, but there is already a noticeable indirect return. Our clients see that we are investing in innovation. Our products and services are using artificial intelligence resources more and more and have high computational capacity. We can resolve computational problems in the Microsoft cloud that no academic institution or governmental agency can. The computational capacity of our cloud is equivalent to several supercomputers. This investment affects our business in a positive way, and the clients understand what we can do.
How is your life these days in Seattle?
My daily life is cool. My wife and I have been lucky: our two children live close by, which is not very common in the USA. My daughter lives 10 minutes from our house, and my son 20 minutes. Last year, my son had twins. As a result, part of our daily life is enjoying our grandchildren. Besides that, Seattle is a great city if you like outdoor sports and nature. My wife really likes hiking and we go on hiking trails. There is also a Brazilian store close by, Kitanda, where I can buy cheese bread, guava paste, and a load of Brazilian products. We miss Brazil, but we go there on vacation every year. I feel very lucky.
