Advances in AI Research, Power of Open-Source Projects and More – Interview with Carlos Escapa, PyTorch, Facebook

We thank Carlos Escapa from PyTorch, Facebook for taking part in this interview.

Academic Background and Specialization

Bala: Could you tell us about your academic background and specialization?

Carlos: My career started with Bachelor’s and Master’s degree in Computer Science. The reason that I got a Master’s degree is that I genuinely enjoyed the university environment and did not want to start working yet. I lived comfortably as a student with internships and part-time jobs. Later I became a specialist in data centre automation, and from that base I evolved into Virtualisation, Cloud Computing, Big Data and Artificial Intelligence.

Recent Advances in AI Research

Bala: What, according to you, are the most exciting advances in AI research in recent years?

Carlos: This is a hard question because AI research is advancing over a broad spectrum of problems.

If I were to choose one specific area, it would be transformers.

We are beginning to see them applied to Natural Language Processing across a vast array of use cases, and there are promising signs that we will see them applied to Computer Vision.

Having said that, AI research continues to move much faster than AI applications; I would like to see more advances in model productisation and governance.

Bala: The recent advances in AI research and easier access to computing resources have made ‘AI in Healthcare’ a promising direction. However, the sensitivity of medical records comes in the way of deploying such AI-powered healthcare solutions at scale. How do you think advances in Privacy-Preserving AI can help us move towards a better healthcare industry?

Carlos: Privacy-preserving refers to a set of mathematical methods that facilitate collaboration between multiple parties while keeping data obfuscated. They have been available for a long time in statistical analysis and can also be used for AI. It can be particularly useful in countries like the US with a highly fragmented healthcare system. As we stand today, it is still difficult to operationalize and requires a lot of ad hoc tooling; this makes it hard to explain to regulators and compliance officers. It is not “low hanging fruit”.

Bala: According to you, what are the industries that could leverage the power of AI and benefit immensely?

Carlos: We have to be conscious that “the power of AI” is not necessarily positive and does not necessarily yield benefits. All technologies, from gunpowder to nuclear fission, have been used both in positive and nefarious ways. I would contend that the public sector is where the biggest benefits of AI could manifest themselves, but it also carries the highest risk because AI can be used to impinge on human rights or drive totalitarianism.

We need to make more advances in AI governance before we can consider how to leverage AI to the fullest.

Essential Skills Beyond Academic Background and Technical Expertise

Bala: As someone who’s working at the intersection of technology, business development and leadership, what skills do you think are sacrosanct, in addition to relevant academic background and technical expertise?

Carlos: Attitude is far more important than skills.

Being open-minded, humble, self-conscious about our limitations and willing to learn every day are sacrosanct.

In the 21st century, more and more time in our careers will be spent experimenting and trying new ideas, and most of them will fail – which is normal. Those who can learn quickly, accept failure and pivot forward to the next experiment will thrive.

Importance of Open-Source Projects in Driving Innovation

Bala: “Open-source projects and frameworks have helped in improving accessibility to Machine Learning and AI Research as well as in the Democratization of Data Science” – Could you please share your views on this dictum? Carlos: Open Source is unquestionably the most powerful force of innovation in the world today, and it applies not only to Data Science but to just about any computer science project and is generalisable to other scientific undertakings.

It enables the top subject matter experts (SMEs) in the world to collaborate in addressing extremely complex problems, and to make solutions available for anyone to use.

Importantly, it also accelerates the transfer of skills from SMEs to collaborators and students, who can see first hand how the SMEs work and contribute to their projects.

It is a force-multiplier without parallel in history.

Bala: How do you think writing can help in the learning process? Do you believe simplifying tech jargons and different algorithms can help in reinforcing understanding?

Carlos: This is a well-known phenomenon that we learn as children: taking notes in class is an effective learning technique and I continue to practice it every day during meetings or video lectures. Summarisation further reinforces the learning – that’s why sending out meeting notes is such a critical task, and not a secretarial function. We have tools that transcribe meetings and NLP algorithms that can provide summarisation, but I choose not to use them.

Advice to Data Science Aspirants

Bala: What would be your advice to aspirants hoping to make a career in Machine Learning and Data Science?

Carlos: My advice is to take courses in the humanities, dig deep on what motivates people, and become experts in the interaction between science and society. Analyse and decompose institutions that changed the course of Science, from the House of Wisdom in Baghdad to CERN in Geneva.

Understand why Newton said that he stood on the shoulders of giants, because that’s what it feels like to be in AI.

Data Science is our youngest science, extending the arc of using mathematics to solve problems that started with Ibn Al-Haythm over 1000 years ago.

Bala: Could you please suggest a few books and learning resources that you think would help aspirants gain foundational skills in Machine Learning and Deep Learning?

Carlos: Foundational skills in ML/DL are best picked up using the plentiful courses available online because they come with exercises and often give direct access to environments where you can try the techniques explained. For instance Microsoft’s “Introduction to PyTorch“. The most popular series is Andrew Ng’s Stanford lectures, available in Coursera.

Rather than suggesting books, and since ML/DL evolves rapidly, I recommend aspirants to listen to recent lectures and keynotes given by top scientists and thinkers in the field who have a broad perspective on the topic, and get in the habit of doing this monthly. Two that I would recommend are Kai-Fu Lee on Innovation and AI, and Soumith Chintala on the future of ML frameworks.