Antoine Caillon

Doctor in deep generative modeling applied to musical signals.

Currently

Research Scientist at Google DeepMind.

PhD Subject

Hierachical temporal learning for multi-instrument and orchestral audio synthesis

Research interests

Neural audio synthesis, real-time implementation of deep generative models, artistic use of artificial intelligence

Education

2020-2023 PhD: Hierachical temporal learning for multi-instrument and orchestral audio synthesis

• ircam (paris)
• Directed by Philippe Esling
• Obtained with High Honours
• PhD manuscript
• Defense
• Actor project

2018-2019 Master Degree 2 ATIAM

• ircam (paris)
• Obtained with High Honours
• Signal filtering, source separation, machine learning, Reactive programming, modelisation and synthesis of different instruments

2017-2018 Master Degree 1 Engineering Sciences

• Sorbonne Université (Paris IV)
• Signal processing, acoustic, mecanic, numeric methods (finite element method, finite difference method)

2015-2017 Mathematic degree

• Sorbonne Université (Paris IV)

2012-2014 Sound engineering degree

• ITEMM, Le Mans

Previous positions

2022 Google Brain, invited student researcher

• Neil Zeghidour, Jesse Engel
• 8-months project, multimodal modeling of raw audio signals
• Speech processing, music processing, large scale modeling
• Work on MusicLM and SingSong

2019 Technicolor, intern

• Alexey Ozerov, Ngoc Q. K. Duong
• 6-month internship on the use of deep learning models to age or de-age speech.
• Deep Learning, Audio processing

Publications

2022 SingSong: Generating musical accompaniments from singing

• Chris Donahue, Antoine Caillon, Adam Roberts et al.
• Preprint

2022 MusicLM: Generating Music From Text

• Preprint

2022 Streamable Neural Audio Synthesis With Non-Causal Convolutions

• Antoine Caillon, Philippe Esling
• DAFx 2020in22

2021 RAVE: A variational autoencoder for fast and high-quality neural audio synthesis

• Antoine Caillon, Philippe Esling
• Preprint

2020 Timbre latent space: exploration and creative aspects

• Antoine Caillon, Adrien Bitton, Brice Gatinet, Philippe Esling
• TIMBRE 2020

2020 Diet deep generative audio models with structured lottery

• Philippe Esling, Ninon Devis, Adrien Bitton, Antoine Caillon, Axel Chemla–Romeu-Santos, Constance Douwes
• DaFX 2020

2019 Assisted Sound Sample Generation with Musical Conditioning in Adversarial Auto-Encoders

• Adrien Bitton, Philippe Esling, Antoine Caillon, Martin Fouilleul
• DaFX 2019

Repositories

2022 nn~ : a Max/MSP external for real-time ai audio processing

• Antoine Caillon
• open source code for the nn~ external

2021-2022 RAVE: Official implementation

• Antoine Caillon
• open source code for the RAVE model

2020 ddsp_pytorch

• Antoine Caillon
• Based on the work from magenta
• Real-time implementation of the DDSP model

Collaborations

2021-2022 ANIMA TM

• Alexander Schubert, in collaboration with Antoine Caillon
• Performed at Centre Georges Pompidou (see description here)

2021 Improvisation, apprentissage profond et fusion d’espace latent

• Maxime Mantovani
• Latent exploration with custom controllers and prior matching techniques

2019-2020 Convergence

• Alexander Schubert, in collaboration with Antoine Caillon, Philippe Esling and Jorge Davila-Chacon
• Awarded with a Golden Nica (Ars Electronica 2020) in Digital arts (see here)

2019-2020 unknown title

• Brice Gatinet, in collaboration with Antoine Caillon
• Creation of wavae, initial work on real-time use of deep learning models

Master class

2022 Neural Audio Synthesis

• Université de Cergy

Neural Audio Synthesis

• Hochschule für Musik und Theater Hamburg

RAVE + nn~

• BERGEN SENTER FOR ELEKTRONISK KUNST

Teaching

2020-now Machine learning project

• Master 2 ATIAM
• 3 months project

2020-now React Native course

• L3 Sorbonne Université