ng'stuff

Making music programmable again

Sat, 08 Feb 2020 00:00:00 GMT

For adepts of retro-gaming who have some foundation in music, retro-gaming music is a nice area to dig into: it has restrictive musicality, which enables to focus on composition and strong auditive identity. But after weeks or even months of research, I came to realize how sub-developed music synthesis could be in interactive systems like video games. This article gives some context about how music used to be made in video games, how game music composition evolved over the years, and eventually opens some doors to new creative horizons. SUMMARY * Game music in the 90’s * Modern music composition * Synthesis from MIDI * Dynamic music synthesis in 2020? GAME MUSIC IN THE 90’S Retro games used to synthesize sound on-the-go by the device’s sound chip or card. This is why retro game music is often referred as chiptune. Many artists still compose chiptune nowadays [http://battleofthebits.org/] - maybe because nostalgia, or maybe because chiptune focuses on composition rather than picking and tweaking instruments, effects and mastering. Anyhow, it’s an amazing discipline for the hobbyist composer! Your browser does not support the audio tag.Quick copycat I made based off Fearofdark’s amazing work ❤️ [https://fearofdark.bandcamp.com/album/the-coffee-zone] Chiptune is often composed with a tracker. In a tracker, audio channels are explicitly separated across vertical tracks, notes are textually sequenced and can each have its own instrument and receive a variety of effects. Trackers are here to allow composition while enforcing control over audio channels. FastTracker2 screenshot [/img/FastTracker2.png]Figure 1. FastTracker 2, a DOS music tracker Although many claim the benefits of the tracker layout because of its compactness, I still think it’s rather outdated and prevents picturing the music well. The main alternative layout, the piano roll, comes closer to music sheets which are the standard of music composition. Notes can intuitively be spanned across time, and music layers can be "drawn". LMMS screenshot [/img/LMMS_PianoRoll.png]Figure 2. Piano roll in LMMS Piano rolls are an essential component of many modern Digital Audio Workstations (DAW) [https://en.wikipedia.org/wiki/Digital_audio_workstation]. Although more visual than trackers, different piano rolls might have many varying features between each other, whereas all trackers are mostly the same. This has an impact on how little standard piano roll based DAWs are. Most trackers produce module files [https://en.wikipedia.org/wiki/Module_file]. These files basically separate the instruments from the notes and effects. A consequence from separating the instruments' sound data is that the module file is usually much smaller than raw audio data. Moreover, isolating instruments formalizes the identity of the music’s sounds. It’s a bit like picking an orchestra! While I love the module format, there are still issues: * Each module file contains its own orchestra * There are many module formats * No module format is close to being standard Because of these, I don’t see any significant advantage in using module files in video games. Instead, there must be some sort of standard in order to separate instruments from music… MODERN MUSIC COMPOSITION Modern video game music seems to have converged with regular "song" music along with the evolution of DAWs. Most games embed compressed raw audio data and it makes sense for most game dev studios that have a dedicated musicians. Producing raw audio gives maximum flexibility in creation: the musician can use and endless variety of instruments, effects, filters, mixes… On top of that, raw audio is very easy to play in the game itself and is, of course, standard. However, for the indie game developer who wants super-dynamic music progressions, for the hobbyist who doesn’t master music that well, or for the studio who wants to enforce strong auditive identity in a game as lean as possible: they have reasons to be frustrated. Indeed, in a developer’s mind: why not making music programmatic? Graphics can easily be, so why not music? There is a standard for having instruments communicate together and with computers: MIDI [https://en.wikipedia.org/wiki/MIDI]. Instead of transferring raw audio, the MIDI protocol consists in transferring simpler abstract signals that will be interpretated according to the receiver. For instance, a MIDI signal can consist of a note with a certain velocity, and the next signal most probably is the release of this note (note off). In this scenario, the same MIDI signals can trigger a very loud note with an echo, a quiet sharp note or even be interpretated as VJ lighting changes! MIDI is the standard abstraction layer for digital music. Signals can be realtime or can be stored in a .mid file (called Standard MIDI File or SMF). This gives us strong hopes for programmatic music! Provided a sound synthesis library, lightweight signals can become music. SYNTHESIS FROM MIDI MIDI now looks like an obvious choice for standard programmatic music. However I had a hard time finding how to actually play it back from a game. Most music learning resources and artists point to proprietary solutions, formats, which work for raw audio but removes programmatic freedom. Polyphone screenshot [/img/Polyphone.png]Figure 3. The Polyphone SoundFont editor The best solutions I found so far rely on SoundFonts [https://en.wikipedia.org/wiki/SoundFont]. Despite being non-standard, they are a well-accepted format to define a bank of instruments. There are some great pieces of software that make great use of SoundFonts for our purpose, notably: * TiMidity [https://github.com/freeors/SDL/blob/master/SDL2_mixer-2.0.1/timidity/timidity.c], a basic midi player originally written for SDL back in the days (1995). It evolved into its separate repositories and can be used as a library, but is very limited * FluidSynth [https://github.com/FluidSynth/fluidsynth], a MIDI synthesis library specifically made to work with SoundFonts and play the music itself. FluidSynth is also easily embeddable in DAWs though a VST [https://en.wikipedia.org/wiki/Virtual_Studio_Technology] These libraries are amazing tracks to get started! But I’m still astonished by the small enthusiasm they grab. A consequence of this is how hard it is to leverage the whole capabilities of FluidSynth in a DAW: most DAWs have their own way to manage effects and thus don’t allow to export those as standard MIDI. Moreover, even if MIDI+Soundfont looks like a "standard" combo, I don’t know any good DAW that allows both MIDI composing and SoundFont creation/edition at the same time. DYNAMIC MUSIC SYNTHESIS IN 2020? Now we reach the point of many wonders. It seems like great music technologies from the 90’s still exist but haven’t reached their full potential. This didn’t prevent music itself to evolve, but what about the way music is used interactively? There are countless video games out there, and aside massive franchises, very few are known for their music. Worse than that: there are so few music games or games that integrate music as part of the gameplay. Despite incredible technology evolutions, I feel like a whole dimension is missing from game development nowadays. For hobbyists, music creativity should be as accessible as visual arts. Music-rich games should be able to fit on a cartridge or a floppy disk like back in the days and should have a well-defined auditive identity. If this article resonates with you too, feel free to share, tweet [https://twitter.com/ngasull/status/1228238628729409536] or reach out to me [/about#contact-me] and I’ll be happy to know what you think and what we can do 🙂 I really believe that the creative community deserves better!

How to simply use vim across multiple files

Sat, 02 Nov 2019 00:00:00 GMT

Vim provides its own unique way to edit code. Discerning what makes it really different helps a lot figuring out why so many developers love vim. This article attempts to summarize these core concepts and provides a reasonable set of tips for an efficient use of vim across multiple files. Nowadays, developers that are invited to vim are being told to install all these cool plugins. Although most of these are very good, I believe plugins make vim’s actual learning curve steeper than it should be. Some plugins disguise vim into vscode, reminding features like file explorer toolbar [https://github.com/scrooloose/nerdtree] or tabs [https://github.com/mihaifm/bufstop]. Because vim is harder to grasp when beginning, limiting the number of plugins is crucial. With its core concepts only, vim already comes with great power and great responsibility. People probably don’t accept this well enough, at least I didn’t and my resulting experience of vim has been years of trying to stick to it and falling back to another editor (or IDE). It’s time to transform this vim hassle into vim power! This article assumes the reader already has basic vim knowledge. When mentioning commands, means Ctrl+key. Capitalization of letters is meaningful as it implies pressing shift at the same time. BUFFERS First of all, it’s time to forget about tabs. In vim, tabs are a power-user feature that may be useful for those who already master buffers and in very specific scenario. Buffers alone are more than enough. Also, no plugin will put vscode’s tabs in vim. Experienced vim users often refer to navigating through files as flowing through buffers. Indeed, when you need to access a piece of code, you just place your cursor in it to explore or edit it. There is no need to care about which buffer is open and where it is. We’ll see that with just a few commands we are able to keep as many buffers open as we like while still keep editing efficiently! OPENING FILES :find path/file opens path/file. :find supports quick file search if we enable it, so let’s run / add to .vimrc the following commands in order to get comfortable: vim set path+=* set wildmenu " Optionally ignore bulky folders: set wildignore+=*/node_modules/* We’ve just set up quick file search! Tab completion now finds files deep in current working directory. For example, :find foobar or :find *.ht will both find some/nested/dir/foobarbaz.html. If multiple files match, the wildmenu we just enabled appears on top on the command line and more presses rotate across matches. EXPLORE FILES Running :find on a folder opens netrw, a simple yet efficient file explorer. It provides a tree view if you hit l a few times! You can also create a new file by hitting %, delete with D and rename with R. Using :find alone goes to current file’s directory and :find . goes to current working directory. NAVIGATING ACROSS BUFFERS A developer’s workflow often involves going back and forth in a code base. Although :find or fzf [https://github.com/junegunn/fzf.vim] are great for quickly opening files, we can go faster once they have already been opened in buffers. * Ctrl+6 swaps to last opened buffer * Ctrl+o moves the cursor to the previous location it last jumped from * Ctrl+i moves the cursor to the next jump The two latter commands take advantage of the jumplist [https://vim.fandom.com/wiki/Jumping_to_previously_visited_locations] which contains all cursor jumps, including inside the same buffer. This is much more powerful than just "tab switching" in other editors as you can extremely quicky jump back and forth more precise contexts. Although they are precise, they might take you more time than a simple Ctrl-6 swap-back or than moving away with a quick :find or fzf. CLOSING BUFFERS It won’t cause any trouble to keep any number of buffers open and it is often recommended to keep them open. However, when we mainly rely on Ctrl-o/Ctrl-i to navigate, we want to keep the jumplist history clean. So if, for example, a file has been opened by mistake, all what is needed is :bw. WINDOWS Windows are viewports on a buffer. A single one takes all vim’s space when launching the editor. Some commands like :help anything spawn a new window. :q closes current window or the editor when no window remain. Windows allow to split the editor, which can be done manually with :split and :vsplit. Window-controlling commands are prefixed by . For example, followed by a direction focuses the nearest window in that direction. As I don’t need editor splits that much, I won’t get more into details. However I run vim from an instance of tmux which allows to seamlessly navigate across split terminals and vim windows [https://github.com/christoomey/vim-tmux-navigator], i.e. with the same shortcut: plus a direction. tmux allows much more and I will write a dedicated article about it. ---------------------------------------- That’s all for file navigation in vim, keeping it simple and mouse-free! I still recommend to look into fzf [https://github.com/junegunn/fzf.vim] for fuzzy finding; I bound it to to speed up file jumps. I also recommend to install ripgrep [https://github.com/BurntSushi/ripgrep] on your system for recursive directory text searches. fzf.vim automatically picks up on ripgrep and you just need to run :Rg textOrRegex to start using it. CHEATSHEET (TL;DR) In buffer Action :find file Open file :find directory Explore directory :find . for root directory :find Explore current file’s directory Swap with previous buffer Jump back Jump next :split Split horizontally :vsplit Split vertically direction Focus nearest window :bw Close buffer :q Close window :help anything Opens related help in a new window. Often more efficient than searching the internet! In netrw Action i Change layout list > details > ls > tree % New file d New directory R Rename file D Delete file With plugins Action Open fzf [https://github.com/junegunn/fzf.vim] Focus nearest window or tmux pane :Rg textOrRegex Recursive text search with ripgrep [https://github.com/BurntSushi/ripgrep]