Music Data Processing and Symbolic Music Generation
with Jingwei Liu
In this workshop participants will have a chance to understand and analyze MIDI metadata as data preprocessing for Neural Networks. We will generate symbolic music using autoregressive neural networks, eg. RNN.
Plan for the First Day
-> date: Saturday July 29
-> format: online in real time in ZOOM, time (4 hours with a break)
-> in short: We will talk about how to do data mining for MIDI files, how to extract information from MIDI metadata using Python in Jupyter notebooks with concrete examples and hands-on programming.
-> start time options: to be decided with the group
option 1) 8 am by SD\Tijuana = 9 pm by Almaty = 3 pm UTC/Europe
option 2) 7 pm by SD\Tijuana = 8 am by Almaty = 2 am UTC/Europe
Plan for the Second Day
-> date: Sunday July 30
-> format: online in real time in ZOOM, time (4 hours with a break)
-> in short: We will use the extracted music information to train the generative neural networks in Pytorch to generate new music pieces. This section will introduce multiple autoregressive neural networks such as RNN, LSTM, etc. to generate music sequences.
-> start time options: to be decided with the group
option 1) 8 am by SD\Tijuana = 9 pm by Almaty = 3 pm UTC/Europe
option 2) 7 pm by SD\Tijuana = 8 am by Almaty = 2 am UTC/Europe
For the next 4 days participants will be assigned with a task, with supervision in Telegram
-> Example task 1 : Programming homework in Jupyter notebooks based on the course materials, in the form of filling in the blanks with guided instructions. The students are supposed to fulfill the homework by themselves and make the codes run correctly as instructed. The answers will be published after the submission.
-> Example task 2 : The students may work in groups, proposing and fulfilling a project together. The project could be on data processing, data analyzing, data mining, or to build a demo neural network for music generation. The results are secondary as there are only a few days, but the experiments, preliminary tests and ideas are important for you to understand the course materials and probably carry on the project by yourselves after the workshop.
Final presentations and review
-> date: Friday 4th of August or Saturday 5th of August (to be decided with the group)
-> format: real-time in ZOOM
Preferable requirements for students
Have basic programming skills in python; have Jupyter notebook installed before the class; maybe have some knowledge on neural networks and Pytorch; have some knowledge on MIDI.
Please check these resources before the workshop: 1) midi to csv ; 2) RNN
Enrollment Closed
Jingwei is currently a Ph.D. student in computer music, at UC San Diego. She got a bachelor's degree in Mathematics from Shanghai Jiao Tong University, China, and a master's in Mathematics from the University of Wisconsin-Madison, United States. Her work is to study humanity subjects (primarily music) using methods developed in quantitative analysis and numerical simulation in the STEM fields. Nowadays as the prevailing of neural networks, her research methods shifted towards neural networks. Practically, she also tries to build high-performance music generation and audio signal processing apps.
Jingwei Liu - theoretical researcher on music and minds. Practitioner at Neural Network programming and music generation.
Neural Networks for Sound Synthesis and Music Generation
with Tornike Karchkhadze
We will look at the world of music generation and sound synthesis using AI technology and explore the latest advancements in the field, considering their implications for creative expression. Through engaging discussions and hands-on exercises, you'll have the opportunity to harness the power of neural networks and unlock new possibilities in music production.
Plan for the First Day
-> date: Saturday August 5
-> format: online in real time in ZOOM, time (4 hours with a break)
-> in short: In the first half, I will provide an introduction to the field of music generation and sound synthesis, covering its history and modern-day tools, including Neural Networks. In the second half, I will talk about my current work at Bose, sharing insights into cutting-edge advancements in this domain.
-> start time: 7 am by SD\Tijuana = 8 pm by Almaty = 2 pm UTC/Europe = 10 am Boston
Plan for the Second Day
-> date: Sunday August 6
-> format: online in real time in ZOOM, time (4 hours with a break)
-> in short: On day two we will dive into practical exercises. I will guide students through the process of using Python for music in the environment of Google Colab, a platform for machine learning. We will look at the tools and have hands-on experience by building the first simple audio/music generation code.
-> start time: 7 am by SD\Tijuana = 8 pm by Almaty = 2 pm UTC/Europe = 10 am Boston
For the next 4 days participants will be assigned with a task, with supervision in Telegram
-> Example task 1 : Building Mozart's "dice game" composition in Python.
-> Example task 2 : Building small Machine Learning models and training them on synthetic data.
-> Example task 3 : Generating FFT from sound in Python with librosa.
Final presentations and review
-> date: Friday 11th of August or Saturday 12th of August (to be decided with the group)
-> format: real-time in ZOOM
Please check these resources before the workshop:
1. McCarthy, John. 2007. "WHAT IS ARTIFICIAL INTELLIGENCE"; Stanford, CA: Stanford University, Nov 12.
2. https://openai.com/blog/jukebox/
3. https://magenta.tensorflow.org/
5. http://repmus.ircam.fr/omax/home
6. https://nsynthsuper.withgoogle.com/
7. https://sites.google.com/eng.ucsd.edu/ucsd-cse-190-summer-2020/syllabus?authuser=0
An audio/music researcher, and data scientist specializing in deep neural sound synthesis techniques and computational creativity research for music/audio generation and composition. Tornike is a second-year Computer Music Ph.D. student in UC San Diego, part of a research group working on reinforcement learning in musical improvisation, and a collaborator on a project about spatial audio HRTF personalization with deep learning.
Tornike Karchkhadze - musician and Machine Learning specialist.
Previously, he earned a Master’s degree in music from the Institute of Sonology, at the Royal Conservatoire in the Hague, Netherlands. He studied at the Sibelius Academy, University of the Arts Helsinki, for two years and received a bachelor's degree in music technology from the Tbilisi State Conservatoire. He also holds a Bachelor of Science in Physics from Tbilisi State University.
Currently, Tornike works at Bose as an Audio ML researcher. He has years of experience in creative musicianship (composition, production) in parallel with years of experience working in statistics, data science, and machine learning.
Worldmaking Jazz 1960s till Now
with Paul Nicholas Roth
This workshop explores sounds and ideas of vanguard improvising musicians from the 1960s until today. These musicians have and continue to expand notions of geography, philosophy, spirituality, history, and futurity through music. We situate “worldmaking” as a site of social justice work, both political and speculative. Musicians explored serve as guides for reimagining past, present, and future.
Plan for the First Day
-> date: Saturday August 12
-> format: online in real time in ZOOM, time (4 hours with a break)
-> in short: Introduction, short lectures, listening sessions, and group discussion. We'll brainstorm ideas for our own worldmaking projects and investigate where these show up in musics we're already interested in.
-> start time: 8 am by SD\Tijuana = 9 pm by Almaty = 3 pm UTC/Europe
Plan for the Second Day
-> date: Sunday August 13
-> format: online in real time in ZOOM, time (4 hours with a break)
-> in short: We will have more lecture material, listening sessions, and extended group discussion. Finally we will workshop ideas for our own speculative worldmaking projects.
-> start time: 8 am by SD\Tijuana = 9 pm by Almaty = 3 pm UTC/Europe
For the next 4 days participants will be assigned with a task, with supervision in Telegram
-> Example task 1 : Get comfortable with deep listening practice and share your experience.
-> Example task 2 : Consider potential worldmaking gestures in your own musical interests and describe them.
-> Example task 3 : Imagine and develop your own speculative worldmaking project.
Final presentations and review
-> date: Friday 18th of August or Saturday 19th of August (to be decided with the group)
-> format: real-time in ZOOM
Preferable requirements for students: "Decent headphones please!"
Please check these resources before the workshop:
Paul Nicholas Roth plays saxophone across popular and experimental music genres, composes, writes scholarship, and does community arts advocacy. Recent sounds are released through earwash records (of which he co-directs—earwash.bandcamp.com); academic projects are published in both Routledge and Indiana University press. He’s currently finishing PhD studies in UCSD’s department of music in the interdisciplinary musicological “Integrative Studies” program. He also curates for the nettnett collective in downtown Tijuana. From 2013-2017 he served on the curatorial team for leading experimental music venue “ausland” in Berlin. He has Masters degrees in both Performance and Musicology from the University of Nevada, Reno, and a Bachelors degree in jazz studies from the University of Miami, Florida.
Paul Nicholas Roth - Musician/scholar/organizer/advocate involved at various registers of music and art.
A Science and Technology Studies Approach to research in Electronic Music
with Pablo Dodero
This class will follow the history of electronic music technology looking at early innovations to modern musical instruments. The aim of this class is to expose students to methodologies that help situate electronic music instruments within their historical and sociopolitical contexts. We may expect to learn about history, musical genres, and key musical pieces surrounding electronic music instruments.
Plan for the First Day
-> date: Saturday August 19
-> format: online in real time in ZOOM, time (4 hours with a break)
-> in short: We will discuss historical electronic music instruments ranging from the 1920's to present day. We will identify key instruments, inventors, and musicians and connect them to musical genres and localities they are associated with. After the break we will look at different texts that discuss music technology from a theoretical perspective. Important names would be:
Trevor Pinch, James Mooney, Don Ihde, Walter Benjamin.
-> start time: 8 am by SD\Tijuana = 9 pm by Almaty = 3 pm UTC/Europe = 9 am Mexico city
Plan for the Second Day
-> date: Sunday August 20
-> format: online in real time in ZOOM, time (4 hours with a break)
-> in short: Each student will choose one instrument, innovator, or genre they wish to research and compile information about its historical context, locality, and person or persons credited to its invention. Students will generate a mock abstract for a paper that discusses this instrument. We will discuss methodologies and theoretical approaches.
-> start time: 8 am by SD\Tijuana = 9 pm by Almaty = 3 pm UTC/Europe = 9 am Mexico city
For the next 4 days participants will be assigned with a task, with supervision in Telegram
-> Example task : Students can expect to be assigned readings, writing prompts, and research exercises to identify primary and secondary sources. There will also be listening sessions and discussions surrounding assigned prompts.
Final presentations and review
-> date: Friday 25th of August
-> format: asynchronous submission in Telegram group of the class with comments
Preferable requirements for students: "Interest in theoretical approaches to sound and music"
