View source for Berkeley Phonetics Machine

The Berkeley Phonetics Machine (BPM) is a virtual machine with software for doing phonetics already installed.

The current BPM derives from the [http://bce.berkeley.edu Berkeley Common Environment (BCE)] machine image, which is used in a variety of classes and workshops on the UC Berkeley campus.

The BPM is described in a [[media:bpm.pdf | paper presented at Interspeech 2016]].

== Requirements ==


In order to use BPM your computer must support virtualization and

# be able to run [https://www.virtualbox.org/ VirtualBox]. Currently VirtualBox supports the running of VMs on host machines with 32-bit and 64-bit Intel-compatible processors, running Windows, OS X, Linux, and Solaris.
# have sufficient disk space (approximately 14GB minimum)
# have sufficient RAM. We recommend you reserve a minimum of 1GB for the virtual machine.
# virtualization support must be enabled in your computer's setup. Our recent experience with Windows machines indicates that virtualization is often present but disabled. For these users the virtual machine will not boot until virtualization is enabled in the system setup. See the [http://bce.berkeley.edu/enabling-virtualization-in-your-pc-bios.html BCE guide to enabling virtualization]. If you are not comfortable making changes in your system setup, consult with someone who is.

== Getting started ==

# Download and install [http://virtualbox.org Virtualbox].
# Download the [https://berkeley.box.com/v/BCE-2016-summer-BPM-1-public 2016-summer-1-public image].
# Open Virtualbox.
# Click File, Import appliance...
# Find the .ova file you downloaded (probably in your Downloads folder), and click Continue.
# In the Appliance Settings dialog box you can change the amount of RAM dedicated to the virtual machine; the default is 1024MB (1GB). If your computer does not have much RAM you might reduce this number *before* you start up the virtual machine. For example, if your computer has only 2GB of RAM, you cannot allocate 2GB to your virtual machine and have enough left for the host computer to continue to run. The amount of RAM you need to leave available to the host computer depends on how much your operating system and open applications need.
# Click Import.

== Summer 2016 public release ==

The [BPM Summer 2016 public release] is ready for use. We recommend you download and use this version.

See the [BPM Summer 2016 preview] page for a description of the no-longer-supported preview image.

== Shared folders ==

Shared folders are folders that reside on the host machine and are accessible to the virtual machine. These are optional but highly recommended. First, it is convenient to be able to access files from both the host and the guest machines. Second, if the guest machine becomes damaged and fails to boot, the files in the shared folder will still be accessible on the host machine. If necessary, you can simply import the BPM again and start working with the data in the shared folder.

These are the steps to create a shared folder:

# Use your file manager to create a folder on your host computer, e.g. <code>bpmshare</code> on your Desktop, or wherever you like.
# Return to Virtualbox, select your virtual machine, and click Settings, Shared Folders.
# Click the + icon to add a shared folder.
# In 'Folder path', browse to the folder you created and choose it.
# Also click 'Auto mount' so that this folder will automatically be available in the virtual machine when you start it.
# Click 'OK' to leave Settings.

When you boot the machine the shared folder will be mounted under <code>/media</code> with an <code>sf_</code> prefix, for example, <code>/media/sf_bpmshare</code>.

== Booting the machine ==

Now you're ready to boot the BPM. Make sure your virtual machine is selected in VirtualBox and click Start. The machine will boot and automatically log in as user 'oski'.

The 'BPM documentation' link on the desktop will open this wiki page.

== Shutting the machine down ==

You can choose to shut the machine down, or you can simply suspend it and save the current state. With the first option, the machine will boot fresh the next time you start it. With the second all of the applications currently open at the time you suspended the machine will return to their previous states.

To shut the machine down, click on the Menu, then the Shutdown icon.

To suspend the machine, take the machine out of fullscreen mode, then click the close button on the host window that encloses the machine. Choose 'Save the machine state' from the dialog box.

== Included software ==

The BPM includes a variety of software packages for doing phonetic research:

# [http://osdoc.cogsci.nl/ OpenSesame] psychology software
# [http://praat.org Praat] phonetic analysis
# [https://github.com/YannickJadoul/Parselmouth Parselmouth library] for Python access to Praat objects and algorithms
# [https://librosa.github.io/librosa/ LibROSA library] for music and audio analysis in Python
# [https://github.com/google/REAPER REAPER], a new pitch tracker by David Talkin (and [http://languagelog.ldc.upenn.edu/nll/?p=17590 recommended by Mark Liberman]) 
# [[audiolabel demo|<code>audiolabel</code>]] - Python library for reading label files, such as Praat textgrids
# [https://github.com/rsprouse/klsyn klsyn] speech synthesis
# [[sox in phonetic research|<code>sox</code>]], 'the Swiss Army knife of sound processing programs'
# [[ffmpeg reference|<code>ffmpeg</code>]], a 'complete, cross-platform solution to record, convert and stream audio and video'
# convertlabel - utility for converting between label file types
# ifcformant - Inverse Filter Control formant tracker
# [https://github.com/rsprouse/espsfree esps utilities], the Entropic Signal Processing System, a set of command line utilities for speech analysis
# wavesurfer, a GUI application for speech analysis, based on ESPS algorithms

=== Partially-included software ===

# [https://www.ling.upenn.edu/phonetics/p2fa/ Penn Phonetics Lab Forced Aligner (p2fa)] - This package is installed but non-functional because it depends on the HTK toolkit, which is not installed in the BPM distributed to the public. See the following section for instructions on how to install HTK with the <code>bpm-update</code> utility.
# display_acq - A Matlab tool for displaying multichannel physiology signals of the type created in the UC Berkeley PhonLab. This requires a Matlab Component Runtime (mcr) that can be installed with <code>bpm-update</code>.
# EdgeTrak - An MS Windows tool for viewing and analyzing ultrasound images. Requires that the <code>wine</code> Windows emulator be installed. In this case either <code>apt-get</code> or <code>bpm-update</code> can do the job. To install the packages necessary for running EdgeTrak open a terminal and give these commands:

 <nowiki>
  sudo bpm-update bpm-update
  sudo bpm-update sources-list
  sudo bpm-update wine      # Use the tab key to select the license agreement.
  sudo bpm-update edgetrak
 </nowiki>

After installation, configure <code>wine</code> by clicking on Applications > Wine > Configure Wine > Graphics tab > check 'Emulate a virtual desktop'.

To run EdgeTrak, open a terminal and give the command:

  wine /opt/Edgetrak/EdgeTrak.exe

== Installing and updating additional packages ==

Since the BPM is based on the [http://ubuntu.com Ubuntu] linux distribution, you can use the [https://help.ubuntu.com/community/AptGet/Howto <code>apt-get</code>] utility to install one of the thousands of [http://packages.ubuntu.com/ packages] that Ubuntu provides.

In addition, the BPM provides the <code>bpm-update</code> utility that performs installation of software packages that may not already be available in Ubuntu and that might be of interest to a subgroup of linguists.

To use the <code>bpm-update</code> utility, do:

  sudo bpm-update <target>

In this command <code><target></code> identifies the package to be installed or updated. To see a list of available <code><target></code>s, do:

  bpm-update --help

An example of a target is <code>htk</code>, which installs the [http://htk.eng.cam.ac.uk/ HTK speech recognition toolkit]. This package is not provided in the BPM distributed to the public because it requires [http://htk.eng.cam.ac.uk/register.shtml user registration]. The command:

  sudo bpm-update htk

prompts for your user registration credentials, then automatically downloads, compiles, and installs the HTK toolkit.

== Versions ==

The current version of BPM is shown in the file <code>version.txt</code> on the machine desktop.

== Sharing with colleagues ==

You can customize your machine and share it with colleagues. Simply shut down the machine, then in VirtualBox choose File, Export appliance.... Choose the machine you wish to export, then click Continue. Provide a filename in the Storage Settings dialog (we recommend that you not overwrite the existing BPM file). Customize the Name and Description fields of the Appliance Settings dialog if you wish, then click Export. The <code>.ova</code> you named will be created, and you can send it to another user.

The <code>.ova</code> file is quite large, probably 3GB or more, and a good way to share it is to use box.net. Berkeley affiliates have access to 50GB of storage on box.net through their Bconnected accounts. Go to [http:/bconnected.berkeley.edu Bconnected] and follow the link to Box.

== Privacy ==

We recommend that you do not use the BPM to access accounts or web sites that you do not wish to share with the world. When you export an appliance to share with others, all of the data stored in the machine is shared. You might not want to share your web history, ssh keys, or other personal information.
Comments on privacy, secrets.

== Enabling symlinks on shared folders ==

From the host system, while the guest is powered off:

    VBoxManage setextradata VM_NAME VBoxInternal2/SharedFoldersEnableSymlinksCreate/SHARE_NAME 1

Use the name of your VM and the shared folder name in place of VM_NAME and SHARE_NAME.

See https://www.virtualbox.org/ticket/10085#comment:14 for an explanation of why this command is necessary.

== Tips and troubleshooting ==

=== Loss of network access ===

Loss of network access occurs frequently in the virtual machine, especially on OS X hosts. This problem tends to happen when the host machine moves between different wifi networks or after the host wakes from sleep. One way to deal with the issue is to restart the virtual machine. You might also be able to avoid the problem altogether by running the following command from the host system, while the guest is powered off:

    VBoxManage modifyvm VM_NAME --natdnshostresolver1 on

Use the name of your VM in place of VM_NAME.

=== Running low on disk space ===

You might run low on disk space in the BPM's root partition after a while. Lots of odd behavior can happen in this event, like failure of tab completion in Terminal windows. You can diagnose the issue with:

   df -h

If the 'Use%' column for '/' (/dev/mapper/BCE--vg-root) is at or near 100 and 'Avail' is at or near 0, then you need to free up some disk space.

A quick and easy place to start is:

   sudo apt-get clean  # clear out apt files found under /var/cache

This will usually free enough space for tab completion in Terminals to start working again.

Another command to try is:

   sudo apt autoremove

This `autoremove` target will cause system packages that are no longer in use to be removed. Sometimes old kernel header packages remain on the system after they are no longer required.

In addition to the kernel headers, you might find old kernel images that are also hanging around and were not removed by `autoremove`. You can check for these kernel image packages with:

   dpkg -l | tail -n +6 | grep -E '^ii\s+linux-image-[0-9]+' | grep -Fv $(uname -r)

That command will find the list of the currently installed kernel image packages, excluding the one that is currently running on your system (you don't want to remove that one!). The command returns either 1) nothing (no packages to remove) or 2) a list that looks something like:

   ii  linux-image-4.4.0-21-generic          4.4.0-21.37       amd64        Linux kernel image for version 4.4.0 on 64 bit x86 SMP
   ii  linux-image-4.4.0-24-generic          4.4.0-24.43       amd64        Linux kernel image for version 4.4.0 on 64 bit x86 SMP
   ii  linux-image-4.4.0-59-generic          4.4.0-59.80       amd64        Linux kernel image for version 4.4.0 on 64 bit x86 SMP

If it returns a list of packages, it is safe to remove them with `sudo apt-get remove --purge <package>`. For the packages in the above list, that would be:

   sudo apt-get remove --purge linux-image-4.4.0-21-generic linux-image-4.4.0-24-generic linux-image-4.4.0-59-generic
   sudo apt autoremove    # remove the headers that go with the image packages you just removed
 
The Anaconda installation of Python can take up a lot of space, and you can often recover some of that by clearing the installation cache:

   conda clean -t

will remove cached package tarballs (used during installation). And this removes unused cached packages (usually because a newer version is installed):

   conda clean -p

If you do not use Python 2.7, you can save more than 1GB of space by removing the `py27` environment:

   conda remove --name py27 --all

Finally, there is a bug that causes old `mkinitramfs` files to hang around in `/var/tmp` even after a reboot of the virtual machine. You can delete these files to reclaim space.