Difference between revisions of "Berkeley Phonetics Machine"

From Phonlab
Jump to navigationJump to search
(24 intermediate revisions by 2 users not shown)
Line 2: Line 2:
   
 
The current BPM derives from the [http://bce.berkeley.edu Berkeley Common Environment (BCE)] machine image, which is used in a variety of classes and workshops on the UC Berkeley campus.
 
The current BPM derives from the [http://bce.berkeley.edu Berkeley Common Environment (BCE)] machine image, which is used in a variety of classes and workshops on the UC Berkeley campus.
  +
  +
The BPM is described in a [[media:bpm.pdf | paper presented at Interspeech 2016]].
   
 
== Requirements ==
 
== Requirements ==
Line 9: Line 11:
   
 
# be able to run [https://www.virtualbox.org/ VirtualBox]. Currently VirtualBox supports the running of VMs on host machines with 32-bit and 64-bit Intel-compatible processors, running Windows, OS X, Linux, and Solaris.
 
# be able to run [https://www.virtualbox.org/ VirtualBox]. Currently VirtualBox supports the running of VMs on host machines with 32-bit and 64-bit Intel-compatible processors, running Windows, OS X, Linux, and Solaris.
# have sufficient disk space (approximately 8GB minimum)
+
# have sufficient disk space (approximately 14GB minimum)
 
# have sufficient RAM. We recommend you reserve a minimum of 1GB for the virtual machine.
 
# have sufficient RAM. We recommend you reserve a minimum of 1GB for the virtual machine.
  +
# virtualization support must be enabled in your computer's setup. Our recent experience with Windows machines indicates that virtualization is often present but disabled. For these users the virtual machine will not boot until virtualization is enabled in the system setup. See the [http://bce.berkeley.edu/enabling-virtualization-in-your-pc-bios.html BCE guide to enabling virtualization]. If you are not comfortable making changes in your system setup, consult with someone who is.
   
 
== Getting started ==
 
== Getting started ==
   
 
# Download and install [http://virtualbox.org Virtualbox].
 
# Download and install [http://virtualbox.org Virtualbox].
# Download the [https://docs.google.com/a/berkeley.edu/uc?id=0B95c5HZgLrhUbUUxMEZ4Y1pvMkk&export=download 2015-spring public virtual machine image].
+
# Download the [https://berkeley.box.com/s/0407f7iwim8cij3g0kmnhlgupguoa76w 2016-2-preview image] (recommended) or [https://docs.google.com/a/berkeley.edu/uc?id=0B95c5HZgLrhUTWNoUlpsOHY3RUE&export=download 2015-spring public virtual machine image]. (If you are a Ling113 student, it is generally better if you use the version distributed by your instructor, which will contain additional software and/or data specific to the course.)
 
# Open Virtualbox.
 
# Open Virtualbox.
 
# Click File, Import appliance...
 
# Click File, Import appliance...
 
# Find the .ova file you downloaded (probably in your Downloads folder), and click Continue.
 
# Find the .ova file you downloaded (probably in your Downloads folder), and click Continue.
# In the Appliance Settings dialog box you can change the amount of RAM dedicated to the virtual machine; the default is 2048MB (2GB). If your computer does not have much RAM you might reduce this number *before* you start up the virtual machine. For example, if your computer has only 2GB of RAM, you cannot allocate 2GB to your virtual machine and have enough left for the host computer to continue to run. The amount of RAM you need to leave available to the host computer depends on how much your operating system and open applications need. 1GB for the host computer is probably the minimum amount.
+
# In the Appliance Settings dialog box you can change the amount of RAM dedicated to the virtual machine; the default is 1024MB (1GB). If your computer does not have much RAM you might reduce this number *before* you start up the virtual machine. For example, if your computer has only 2GB of RAM, you cannot allocate 2GB to your virtual machine and have enough left for the host computer to continue to run. The amount of RAM you need to leave available to the host computer depends on how much your operating system and open applications need.
 
# Click Import.
 
# Click Import.
   
Previous versions:
+
Previous machine images:
   
 
[http://linguistics.berkeley.edu/vm/bpm.20141015.ova 2014 image]
 
[http://linguistics.berkeley.edu/vm/bpm.20141015.ova 2014 image]
  +
  +
== Summer 2016 preview ==
  +
  +
A preview machine image for Summer 2016 is available. [[BPM Summer 2016 | Take a look]] at some of the updates.
   
 
== Shared folders ==
 
== Shared folders ==
Line 73: Line 80:
 
=== Partially-included software ===
 
=== Partially-included software ===
   
# [https://www.ling.upenn.edu/phonetics/p2fa/ Penn Phonetics Lab Forced Aligner (p2fa)] - This package is installed but non-functional because it depends on the HTK toolkit, which is not installed in the BPM distributed to the public. See the section for instructions on how to install HTK.
+
# [https://www.ling.upenn.edu/phonetics/p2fa/ Penn Phonetics Lab Forced Aligner (p2fa)] - This package is installed but non-functional because it depends on the HTK toolkit, which is not installed in the BPM distributed to the public. See the following section for instructions on how to install HTK with the <code>bpm-update</code> utility.
  +
# display_acq - A Matlab tool for displaying multichannel physiology signals of the type created in the UC Berkeley Phonology Lab. This requires a Matlab Component Runtime (mcr) that can be installed with <code>bpm-update</code>.
  +
# EdgeTrak - An MS Windows tool for viewing and analyzing ultrasound images. Requires that the <code>wine</code> Windows emulator be installed. In this case either <code>apt-get</code> or <code>bpm-update</code> can do the job. To install the packages necessary for running EdgeTrak open a terminal and give these commands:
  +
  +
<nowiki>
  +
sudo bpm-update bpm-update
  +
sudo bpm-update sources-list
  +
sudo bpm-update wine # Use the tab key to select the license agreement.
  +
sudo bpm-update edgetrak
  +
</nowiki>
  +
  +
After installation, configure <code>wine</code> by clicking on Applications > Wine > Configure Wine > Graphics tab > check 'Emulate a virtual desktop'.
  +
  +
To run EdgeTrak, open a terminal and give the command:
  +
  +
wine /opt/Edgetrak/EdgeTrak.exe
   
 
== Installing and updating additional packages ==
 
== Installing and updating additional packages ==
Line 89: Line 111:
 
bpm-update --help
 
bpm-update --help
   
An example of a target is <code>htk</code>, which installs the [http://htk.eng.cam.ac.uk/ HTK speech recognition toolkit]. This package is not provided in the BPM distributed to the public because it requires user registration. The command:
+
An example of a target is <code>htk</code>, which installs the [http://htk.eng.cam.ac.uk/ HTK speech recognition toolkit]. This package is not provided in the BPM distributed to the public because it requires [http://htk.eng.cam.ac.uk/register.shtml user registration]. The command:
   
 
sudo bpm-update htk
 
sudo bpm-update htk
Line 109: Line 131:
 
We recommend that you do not use the BPM to access accounts or web sites that you do not wish to share with the world. When you export an appliance to share with others, all of the data stored in the machine is shared. You might not want to share your web history, ssh keys, or other personal information.
 
We recommend that you do not use the BPM to access accounts or web sites that you do not wish to share with the world. When you export an appliance to share with others, all of the data stored in the machine is shared. You might not want to share your web history, ssh keys, or other personal information.
 
Comments on privacy, secrets.
 
Comments on privacy, secrets.
  +
  +
== Enabling symlinks on shared folders ==
  +
  +
From the host system, while the guest is powered off:
  +
  +
VBoxManage setextradata VM_NAME VBoxInternal2/SharedFoldersEnableSymlinksCreate/SHARE_NAME 1
  +
  +
Use the name of your VM and the shared folder name in place of VM_NAME and SHARE_NAME.
  +
  +
See https://www.virtualbox.org/ticket/10085#comment:14 for an explanation of why this command is necessary.
  +
  +
== Tips and troubleshooting ==
  +
  +
=== Loss of network access ===
  +
  +
Loss of network access occurs frequently in the virtual machine, especially on OS X hosts. This problem tends to happen when the host machine moves between different wifi networks or after the host wakes from sleep. One way to deal with the issue is to restart the virtual machine. You might also be able to avoid the problem altogether by running the following command from the host system, while the guest is powered off:
  +
  +
VBoxManage modifyvm VM_NAME --natdnshostresolver1 on
  +
  +
Use the name of your VM in place of VM_NAME.
  +
  +
=== Running low on disk space ===
  +
  +
You might run low on disk space in the BPM's root partition after a while. Lots of odd behavior can happen in this event, like failure of tab completion in Terminal windows. You can diagnose the issue with:
  +
  +
df -h
  +
  +
If the 'Use%' column for '/' (/dev/mapper/BCE--vg-root) is at or near 100 and 'Avail' is at or near 0, then you need to free up some disk space.
  +
  +
A quick and easy place to start is:
  +
  +
sudo apt-get clean # clear out apt files found under /var/cache
  +
  +
This will usually free enough space for tab completion in Terminals to start working again.
  +
  +
Another command to try is:
  +
  +
sudo apt autoremove
  +
  +
This `autoremove` target will cause system packages that are no longer in use to be removed. Sometimes old kernel header packages remain on the system after they are no longer required.
  +
  +
Finally, there is a bug that causes old `mkinitramfs` files to hang around in `/var/tmp` even after a reboot of the virtual machine. You can delete these files to reclaim space.

Revision as of 15:38, 17 July 2017

The Berkeley Phonetics Machine (BPM) is a virtual machine with software for doing phonetics already installed.

The current BPM derives from the Berkeley Common Environment (BCE) machine image, which is used in a variety of classes and workshops on the UC Berkeley campus.

The BPM is described in a paper presented at Interspeech 2016.

Requirements

In order to use BPM your computer must support virtualization and

  1. be able to run VirtualBox. Currently VirtualBox supports the running of VMs on host machines with 32-bit and 64-bit Intel-compatible processors, running Windows, OS X, Linux, and Solaris.
  2. have sufficient disk space (approximately 14GB minimum)
  3. have sufficient RAM. We recommend you reserve a minimum of 1GB for the virtual machine.
  4. virtualization support must be enabled in your computer's setup. Our recent experience with Windows machines indicates that virtualization is often present but disabled. For these users the virtual machine will not boot until virtualization is enabled in the system setup. See the BCE guide to enabling virtualization. If you are not comfortable making changes in your system setup, consult with someone who is.

Getting started

  1. Download and install Virtualbox.
  2. Download the 2016-2-preview image (recommended) or 2015-spring public virtual machine image. (If you are a Ling113 student, it is generally better if you use the version distributed by your instructor, which will contain additional software and/or data specific to the course.)
  3. Open Virtualbox.
  4. Click File, Import appliance...
  5. Find the .ova file you downloaded (probably in your Downloads folder), and click Continue.
  6. In the Appliance Settings dialog box you can change the amount of RAM dedicated to the virtual machine; the default is 1024MB (1GB). If your computer does not have much RAM you might reduce this number *before* you start up the virtual machine. For example, if your computer has only 2GB of RAM, you cannot allocate 2GB to your virtual machine and have enough left for the host computer to continue to run. The amount of RAM you need to leave available to the host computer depends on how much your operating system and open applications need.
  7. Click Import.

Previous machine images:

2014 image

Summer 2016 preview

A preview machine image for Summer 2016 is available. Take a look at some of the updates.

Shared folders

Shared folders are folders that reside on the host machine and are accessible to the virtual machine. These are optional but highly recommended. First, it is convenient to be able to access files from both the host and the guest machines. Second, if the guest machine becomes damaged and fails to boot, the files in the shared folder will still be accessible on the host machine. If necessary, you can simply import the BPM again and start working with the data in the shared folder.

These are the steps to create a shared folder:

  1. Use your file manager to create a folder on your host computer, e.g. bpmshare on your Desktop, or wherever you like.
  2. Return to Virtualbox, select your virtual machine, and click Settings, Shared Folders.
  3. Click the + icon to add a shared folder.
  4. In 'Folder path', browse to the folder you created and choose it.
  5. Also click 'Auto mount' so that this folder will automatically be available in the virtual machine when you start it.
  6. Click 'OK' to leave Settings.

When you boot the machine the shared folder will be mounted under /media with an sf_ prefix, for example, /media/sf_bpmshare.

Booting the machine

Now you're ready to boot the BPM. Make sure your virtual machine is selected in VirtualBox and click Start. The machine will boot and automatically log in as user 'oski'.

The 'BPM documentation' link on the desktop will open this wiki page.

Shutting the machine down

You can choose to shut the machine down, or you can simply suspend it and save the current state. With the first option, the machine will boot fresh the next time you start it. With the second all of the applications currently open at the time you suspended the machine will return to their previous states.

To shut the machine down, click on the Menu, then the Shutdown icon.

To suspend the machine, take the machine out of fullscreen mode, then click the close button on the host window that encloses the machine. Choose 'Save the machine state' from the dialog box.

Included software

The BPM includes a variety of software packages for doing phonetic research:

  1. OpenSesame psychology software
  2. Praat phonetic analysis
  3. wavesurfer
  4. klsyn speech synthesis
  5. sox
  6. ffmpeg
  7. convertlabel - utility for converting between label file types
  8. audiolabel - Python library for reading label files, such as Praat textgrids
  9. ifcformant - Inverse Filter Control formant tracker
  10. make_text_grids
  11. esps utilities

Partially-included software

  1. Penn Phonetics Lab Forced Aligner (p2fa) - This package is installed but non-functional because it depends on the HTK toolkit, which is not installed in the BPM distributed to the public. See the following section for instructions on how to install HTK with the bpm-update utility.
  2. display_acq - A Matlab tool for displaying multichannel physiology signals of the type created in the UC Berkeley Phonology Lab. This requires a Matlab Component Runtime (mcr) that can be installed with bpm-update.
  3. EdgeTrak - An MS Windows tool for viewing and analyzing ultrasound images. Requires that the wine Windows emulator be installed. In this case either apt-get or bpm-update can do the job. To install the packages necessary for running EdgeTrak open a terminal and give these commands:
  sudo bpm-update bpm-update
  sudo bpm-update sources-list
  sudo bpm-update wine      # Use the tab key to select the license agreement.
  sudo bpm-update edgetrak
 

After installation, configure wine by clicking on Applications > Wine > Configure Wine > Graphics tab > check 'Emulate a virtual desktop'.

To run EdgeTrak, open a terminal and give the command:

 wine /opt/Edgetrak/EdgeTrak.exe

Installing and updating additional packages

Since the BPM is based on the Ubuntu linux distribution, you can use the apt-get utility to install one of the thousands of packages that Ubuntu provides.

In addition, the BPM provides the bpm-update utility that performs installation of software packages that may not already be available in Ubuntu and that might be of interest to a subgroup of linguists.

To use the bpm-update utility, do:

 sudo bpm-update <target>

In this command <target> identifies the package to be installed or updated. To see a list of available <target>s, do:

 bpm-update --help

An example of a target is htk, which installs the HTK speech recognition toolkit. This package is not provided in the BPM distributed to the public because it requires user registration. The command:

 sudo bpm-update htk

prompts for your user registration credentials, then automatically downloads, compiles, and installs the HTK toolkit.

Versions

The current version of BPM is shown in the file version.txt on the machine desktop.

Sharing with colleagues

You can customize your machine and share it with colleagues. Simply shut down the machine, then in VirtualBox choose File, Export appliance.... Choose the machine you wish to export, then click Continue. Provide a filename in the Storage Settings dialog (we recommend that you not overwrite the existing BPM file). Customize the Name and Description fields of the Appliance Settings dialog if you wish, then click Export. The .ova you named will be created, and you can send it to another user.

The .ova file is quite large, probably 3GB or more, and a good way to share it is to use box.net. Berkeley affiliates have access to 50GB of storage on box.net through their Bconnected accounts. Go to [http:/bconnected.berkeley.edu Bconnected] and follow the link to Box.

Privacy

We recommend that you do not use the BPM to access accounts or web sites that you do not wish to share with the world. When you export an appliance to share with others, all of the data stored in the machine is shared. You might not want to share your web history, ssh keys, or other personal information. Comments on privacy, secrets.

Enabling symlinks on shared folders

From the host system, while the guest is powered off:

   VBoxManage setextradata VM_NAME VBoxInternal2/SharedFoldersEnableSymlinksCreate/SHARE_NAME 1

Use the name of your VM and the shared folder name in place of VM_NAME and SHARE_NAME.

See https://www.virtualbox.org/ticket/10085#comment:14 for an explanation of why this command is necessary.

Tips and troubleshooting

Loss of network access

Loss of network access occurs frequently in the virtual machine, especially on OS X hosts. This problem tends to happen when the host machine moves between different wifi networks or after the host wakes from sleep. One way to deal with the issue is to restart the virtual machine. You might also be able to avoid the problem altogether by running the following command from the host system, while the guest is powered off:

   VBoxManage modifyvm VM_NAME --natdnshostresolver1 on

Use the name of your VM in place of VM_NAME.

Running low on disk space

You might run low on disk space in the BPM's root partition after a while. Lots of odd behavior can happen in this event, like failure of tab completion in Terminal windows. You can diagnose the issue with:

  df -h

If the 'Use%' column for '/' (/dev/mapper/BCE--vg-root) is at or near 100 and 'Avail' is at or near 0, then you need to free up some disk space.

A quick and easy place to start is:

  sudo apt-get clean  # clear out apt files found under /var/cache

This will usually free enough space for tab completion in Terminals to start working again.

Another command to try is:

  sudo apt autoremove

This `autoremove` target will cause system packages that are no longer in use to be removed. Sometimes old kernel header packages remain on the system after they are no longer required.

Finally, there is a bug that causes old `mkinitramfs` files to hang around in `/var/tmp` even after a reboot of the virtual machine. You can delete these files to reclaim space.