Berkeley Phonetics Machine

From Phonlab
Revision as of 08:49, 22 January 2021 by Ronald (talk | contribs) (→‎Installing and updating additional packages)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

The Berkeley Phonetics Machine (BPM) is a virtual machine with software for doing phonetics already installed.

The current BPM derives from the Berkeley Common Environment (BCE) machine image, which is used in a variety of classes and workshops on the UC Berkeley campus.

The BPM is described in a paper presented at Interspeech 2016.

Requirements

In order to use BPM your computer must support virtualization and

  1. be able to run VirtualBox. Currently VirtualBox supports the running of VMs on host machines with 32-bit and 64-bit Intel-compatible processors, running Windows, OS X, Linux, and Solaris.
  2. have sufficient disk space (approximately 14GB minimum)
  3. have sufficient RAM. We recommend you reserve a minimum of 1GB for the virtual machine.
  4. virtualization support must be enabled in your computer's setup. Our recent experience with Windows machines indicates that virtualization is often present but disabled. For these users the virtual machine will not boot until virtualization is enabled in the system setup. See the BCE guide to enabling virtualization. If you are not comfortable making changes in your system setup, consult with someone who is.

Getting started

  1. Download and install Virtualbox. Important note for MacOS: Newer versions of MacOS require you to update your security settings within 30 minutes of installing VirtualBox. If you don't perform this step in time your virtual machine will not boot, and you will have to reinstall VirtualBox.
  2. Download the 2016-summer-1-public image.
  3. Open Virtualbox.
  4. Click File, Import appliance...
  5. Find the .ova file you downloaded (probably in your Downloads folder), and click Continue.
  6. In the Appliance Settings dialog box you can change the amount of RAM dedicated to the virtual machine; the default is 1024MB (1GB). If your computer does not have much RAM you might reduce this number *before* you start up the virtual machine. For example, if your computer has only 2GB of RAM, you cannot allocate 2GB to your virtual machine and have enough left for the host computer to continue to run. The amount of RAM you need to leave available to the host computer depends on how much your operating system and open applications need.
  7. Click Import.

Summer 2016 public release

The 2016-summer-1-public release is the officially supported version. We recommend you download and use this version.

See the expired BPM Summer 2016 preview page for a description of the no-longer-supported preview image.

Default username

The default user is 'ubuntu'. Commands that require elevated privileges can be executed via sudo and without a password.

BPM version

The BPM version is stored as a string in the BPM_VERSION environment variable.

Shared folders

Shared folders are folders that reside on the host machine and are accessible to the virtual machine. These are optional but highly recommended. First, it is convenient to be able to access files from both the host and the guest machines. Second, if the guest machine becomes damaged and fails to boot, the files in the shared folder will still be accessible on the host machine. If necessary, you can simply import the BPM again and start working with the data in the shared folder.

These are the steps to create a shared folder:

  1. Use your file manager to create a folder on your host computer, e.g. bpmshare on your Desktop, or wherever you like.
  2. Return to Virtualbox, select your virtual machine, and click Settings, Shared Folders.
  3. Click the + icon to add a shared folder.
  4. In 'Folder path', browse to the folder you created and choose it.
  5. Also click 'Auto mount' so that this folder will automatically be available in the virtual machine when you start it.
  6. Click 'OK' to leave Settings.

When you boot the machine the shared folder will be mounted under /media with an sf_ prefix, for example, /media/sf_bpmshare.

Booting the machine

Now you're ready to boot the BPM. Make sure your virtual machine is selected in VirtualBox and click Start. The machine will boot and automatically log in as user 'ubuntu'.

Shutting the machine down

You can choose to shut the machine down, or you can simply suspend it and save the current state. With the first option, the machine will boot fresh the next time you start it. With the second all of the applications currently open at the time you suspended the machine will return to their previous states.

To shut the machine down, click on the Menu, then the Shutdown icon.

To suspend the machine, take the machine out of fullscreen mode, then click the close button on the host window that encloses the machine. Choose 'Save the machine state' from the dialog box.

Included software

The BPM includes a variety of software packages for doing phonetic research:

  1. OpenSesame psychology software
  2. Anaconda Python 3.7 (and Python 3.5 in a separate Anaconda environment)
  3. The JupyterLab interface for data exploration and documentation
  4. The Montreal Forced Aligner, including pretrained models for English and Mandarin
  5. Praat phonetic analysis
  6. Parselmouth library for Python access to Praat objects and algorithms
  7. LibROSA library for music and audio analysis in Python
  8. REAPER, a new pitch tracker by David Talkin (and recommended by Mark Liberman)
  9. audiolabel - Python library for reading label files, such as Praat textgrids
  10. klsyn speech synthesis
  11. sox, 'the Swiss Army knife of sound processing programs'
  12. ffmpeg, a 'complete, cross-platform solution to record, convert and stream audio and video'
  13. convertlabel - utility for converting between label file types
  14. ifcformant - Inverse Filter Control formant tracker
  15. esps utilities, the Entropic Signal Processing System, a set of command line utilities for speech analysis
  16. wavesurfer, a GUI application for speech analysis, based on ESPS algorithms

Notes on important packages

Python

Anaconda Python 3.7 is the default Python and 2.x is not installed. Python 3.5 is installed in an Anaconda environment named py35. It's primary purpose is to be the environment in which OpenSesame runs (for complicated packaging reasons it is not currently feasible to install OpenSesame under Anaconda Python 3.7). To activate this environment in a Terminal window, do: 

 source activate py35

It is not necessary to explicitly activate the py35 environment in order to run OpenSesame.

Montreal Forced Aligner

The Montreal Forced Aligner is installed under /opt/montreal-forced-aligner, and symlinks to its executable files (mfa_*) are created in /usr/local/bin. The English and Mandarin pretrained models are in /opt/montreal-forced-aligner/pretrained_models.

Opensesame

The version of Opensesame is 3.2. Inline scripts use Python 3.5, and the expyriment and psychopy backends are not available. Look for it under Applications > Programming or run opensesame at the command line.

JupyterLab and Jupyter notebooks

You can create Jupyter notebooks as in previous versions of the BPM, as well as in the new JupyterLab interace, which adds more features to the notebook environment. To run JupyterLab, do: 

 jupyter-lab

One notable new feature in JupyterLab is the variable inspector. To enable it, first launch JupyterLab, then click on the palette icon for a list of commands. Click on the 'Open Variable Inspector' command.

For the older notebook behavior, do: 

 jupyter notebook

Partially-included software

  1. Penn Phonetics Lab Forced Aligner (p2fa) - This package is installed but non-functional because it depends on the HTK toolkit, which is not installed in the BPM distributed to the public. See the following section for instructions on how to install HTK with the bpm-update utility.
  2. EdgeTrak - An MS Windows tool for viewing and analyzing ultrasound images. Requires that the wine Windows emulator be installed. In this case either apt-get or bpm-update can do the job. To install the packages necessary for running EdgeTrak open a terminal and give these commands:
  sudo bpm-update bpm-update
  sudo bpm-update sources-list
  sudo bpm-update wine      # Use the tab key to select the license agreement.
  sudo bpm-update edgetrak

After installation, configure wine by clicking on Applications > Wine > Configure Wine > Graphics tab > check 'Emulate a virtual desktop'.

To run EdgeTrak, open a terminal and give the command: 

 wine /opt/Edgetrak/EdgeTrak.exe

Installing and updating additional packages

Since the BPM is based on the Ubuntu linux distribution, you can use the apt-get utility to install one of the thousands of packages that Ubuntu provides.

In addition, the BPM provides the bpm-update utility that performs installation of software packages that may not already be available in Ubuntu and that might be of interest to a subgroup of linguists.

To use the bpm-update utility, do: 

 sudo bpm-update <target>

In this command <target> identifies the package to be installed or updated. To see a list of available <target>s, do: 

 bpm-update --help

An example of a target is htk, which installs the HTK speech recognition toolkit. This package is not provided in the BPM distributed to the public because it requires user registration. The command: 

 sudo bpm-update htk

prompts for your user registration credentials, then automatically downloads, compiles, and installs the HTK toolkit.

Installing FieldWorks (FLEx)

Field researchers can install FieldWorks (FLEx) in the BPM, and this is one way that Mac users can use FieldWorks, which runs only on Windows and Linux. To install, open a terminal and do: 

 sudo bpm-update bpm-update
 sudo bpm-update apt-get
 sudo bpm-update fieldworks

At one point during the install you will be asked to accept a license agreement. To accept, press the Tab key to navigate until the 'OK' button is highlighted, then press Enter. If you press too many times and pass 'OK', use Shift-Tab to go back, or keep pressing Tab until 'OK' is highlighted again. The license screen appears as follows:

Error creating thumbnail: Unable to save thumbnail to destination

Check to see whether any error messages appear in the installation output. Assuming no errors are found:

  1. Power off the BPM. The power button is in the upper right corner.
  2. (Recommended but optional if your host machine does not have adequate memory or CPU cores): In VirtualBox, select your machine and click 'Settings' > 'System'.
    1. Under 'Motherboard' ensure at least 2048MB (2GB) of memory is allocated to the machine.
    2. Under 'Processor' select 2 CPUs.
    3. Click 'OK' to save the settings.
  3. Start the BPM.

Once the BPM has restarted, look for the application in the desktop menu under Applications > Education > Fieldworks Language Explorer.

Sharing with colleagues

You can customize your machine and share it with colleagues. Simply shut down the machine, then in VirtualBox choose File, Export appliance.... Choose the machine you wish to export, then click Continue. Provide a filename in the Storage Settings dialog (we recommend that you not overwrite the existing BPM file). Customize the Name and Description fields of the Appliance Settings dialog if you wish, then click Export. The .ova you named will be created, and you can send it to another user.

The .ova file is quite large, probably 3GB or more, and a good way to share it is to use box.net. Berkeley affiliates have access to 50GB of storage on box.net through their Bconnected accounts. Go to [http:/bconnected.berkeley.edu Bconnected] and follow the link to Box.

Privacy

We recommend that you do not use the BPM to access accounts or web sites that you do not wish to share with the world. When you export an appliance to share with others, all of the data stored in the machine is shared. You might not want to share your web history, ssh keys, or other personal information. Comments on privacy, secrets.

Enabling symlinks on shared folders

From the host system, while the guest is powered off: 

   VBoxManage setextradata VM_NAME VBoxInternal2/SharedFoldersEnableSymlinksCreate/SHARE_NAME 1

Use the name of your VM and the shared folder name in place of VM_NAME and SHARE_NAME.

See https://www.virtualbox.org/ticket/10085#comment:14 for an explanation of why this command is necessary.

Tips and troubleshooting

Loss of network access

Loss of network access occurs frequently in the virtual machine, especially on OS X hosts. This problem tends to happen when the host machine moves between different wifi networks or after the host wakes from sleep. One way to deal with the issue is to restart the virtual machine. You might also be able to avoid the problem altogether by running the following command from the host system, while the guest is powered off: 

   VBoxManage modifyvm VM_NAME --natdnshostresolver1 on

Use the name of your VM in place of VM_NAME.

Running low on disk space

You might run low on disk space in the BPM's root partition after a while. Lots of odd behavior can happen in this event, like failure of tab completion in Terminal windows. You can diagnose the issue with: 

  df -h

If the 'Use%' column for '/' (/dev/mapper/BCE--vg-root) is at or near 100 and 'Avail' is at or near 0, then you need to free up some disk space.

The easy way to free up some disk space is with bpm-update: 

  sudo bpm-update recover_disk_space

This command executes a number of steps to reduce disk usage by various software package installers and helps in many situations. Larger datasets or software you install yourself could also fill the root partition, and bpm-update does not reduce those types of disk usage.

How to enable the IPA keyboard

A keyboard for use with the SIL Charis/Doulos fonts is installed. If you want to use this keyboard, first enable the Keyboard Input Methods: 

 Applications > Settings Manager > IBus Preferences (under Other)

Answer 'Yes' to start the Keyboard Input Methods (IBus Daemon). The IBus Prefences setup window will show that an IPA keyboard is available on the Input Methods tab. You can close the preferences window after the daemon has been started.

To switch between keyboards, click on the input method switcher in the top right corner of the desktop, on the menu bar near the clock applet. Select the keyboard you wish to use. US English and IPA are already installed, and you can add others with IBus Preferences.

Retrieved from "http://linguistics.berkeley.edu/plab/guestwiki/index.php?title=Berkeley_Phonetics_Machine&oldid=1107"