Computational Mathematics

Adventures in Computing, Statistics, and R

R and RStudio incompatibility with Yosemite Mac OS X 10.10

9 Comments »

There is currently a bug (or feature?) in the current version of Yosemite (OS X 10.10) that messes with the passing of environmental variables to programs launched from Finder (as pointed out by Adam Maxwell).

Notably, this means that PATH variables are not passed properly to R.app or RStudio.app. You may end up seeing errors such as the following:

Error in system("pdflatex", intern = TRUE) : error in running command
sh: pdflatex: command not found

Error in system("convert", intern = TRUE) : error in running command
sh: convert: command not found

Until Apple releases a fix to this, the easy workaround is to launch the desired application from the command-line (terminal).

For example to launch R, you can instead run

/Applications/R.app/Contents/MacOS/R

And for RStudio, you can run

/Applications/RStudio.app/Contents/MacOS/RStudio

Let’s hope Apple releases a fix soon.

GTA R Users Group – Using R for Data Mining Competitions

10 Comments »

Here are the presentation slides I used for my talk on “Using R for Data Mining Competitions” at Ryerson University as part of the Greater Toronto Area (GTA) R User’s Meetup Group.

Special thanks to Anthony Goldbloom from Kaggle and various competition winners for sharing their code and winning solutions. Also, thanks to Leo Guelman and Murtaza Haider for organizing the event!

Please don’t hesitate to contact me if you have any questions.

Installing Rmpi with OpenMPI on Mac OS X Lion

9 Comments »

For whatever reason, Apple decided not to include OpenMPI in Mac OS X Lion (it was supported in Leopard and Snow Leopard). I found this out the hard way after doing a clean install of Lion.

Here are steps to install OpenMPI and get it working with the Rmpi package in R. One benefit of doing it this way is that you get the latest version of OpenMPI.

Step 1 – Install MacPorts
MacPorts is a great tool to compile and install many packages with ease, including OpenMPI. Install MacPorts by following the instructions on this page. Note that you will need to install Xcode off the App Store first.

Step 2 – Install OpenMPI on MacPorts
At the time of this posting, according to this ticket there is a problem compiling OpenMPI on Lion. To get around this, append the following block of code to: /opt/local/var/macports/sources/rsync.macports.org/release/ports/science/openmpi/Portfile

if {${configure.compiler} == "llvm-gcc-4.2"} {
configure.compiler clang
}

Update: This issue has been fixed.
then in Terminal, run:

sudo port install openmpi

Note that gcc44 and openmpi will take a while to build (several hours on my Macbook Air).

Step 3 – Compile and Install Rmpi
It’s best to compile the latest version from source. Download the .tar.gz file from the Rmpi site (at the time of this post, the latest version is 0.6-0 on the developer branch). Then in Terminal, install the package pointing to the proper OpenMPI directory paths with:

R CMD INSTALL --configure-args="--with-Rmpi-include=/opt/local/include/openmpi --with-Rmpi-libpath=/opt/local/lib --with-Rmpi-type=OPENMPI" Rmpi_0.6-0.tar.gz

You can try to load the library in R and run a few test lines to make sure everything is configured properly.

library(Rmpi)
mpi.spawn.Rslaves(nslaves=2)
mpi.remote.exec(runif(5))
mpi.close.Rslaves()

R with Vim on Mac OS X

4 Comments »

The built-in script editor for the Mac OS X R GUI actually isn’t bad. In fact it is much better than its Window’s counterpart. In particular, it has:

  • Syntax highlighting
  • Auto-completion
  • Bracket matching
  • Auto-indent
  • Block code execution (with blocked command history)

However, when coding in R and pretty much any other language, Vim has always been my goto text editor of choice. Once you get past the fairly steep learning curve, nothing comes close to it in terms of coding efficiency and navigation except perhaps Emacs, but let’s not go there.

Michael Bojanowski wrote a blog post on using R with Vim on an Ubuntu machine. One commenter asked about getting it to work on Mac OS X.

First of all, I highly recommend the MacVim port of Vim (not to be confused with macvim.org). You can get the latest version (7.3.53 at time of post) at github:

https://github.com/b4winckler/macvim/downloads

The Vim-R-plugin that Michal mentioned works on OS X with the Screen plugin as well. However, a simpler method is to use one of the following two OS X specific scripts that rely on AppleScript.

R.vim
R.vim uses AppleScript to send selected lines of code to an R buffer. Simply hit <F3> to run the selected lines in R.  If R is not open, it will automatically open a new session. Note that it opens R.app instead of R64.app so if you need to run the 64-bit version you’ll have to tinker with the script a bit.

R-MacOSX
R-MacOSX is also pretty straight forward, although I couldn’t get it working properly. It defaults to <Cmd-E> to run selected lines and <Shift-Cmd-E> to source the entire file.

In either case, key mappings are customizable but personally, I have no trouble with <F3> to execute code using R.vim.  Neither are as full-featured as Vim-R-plugin but I find them sufficient as all I really need is to send blocks of code to R.

Happy Vimming!

R on the iPhone

6 Comments »

iPhone users who have jailbroken their phone have been able to compile R for a while now since they have access to the command line via MobileTerminal.  However, I just recently found out that a package is now up on Cydia to ease installation. So I fired up my iPhone 4 and installed R on it just for fun using the instructions here.

I’ll write out the complete steps for installing R onto the iPhone here as they are actually quite simple:

1. Have a jailbroken iPhone and the Cydia package manager app installed
(instructions not provided here).

2. Install OpenSSH and MobileTerminal via Cydia
Note that if you are running iOS 4, the current version of MobileTerminal on Cydia does not work properly, you’ll need to get an updated version from:

MobileTerminal 354.3-12 Debian package

Copy the file to your iPhone via SFTP (or download it to your iPhone directly) and then install it via SSH using:

dpkg -i mobileterminal_364.3-12_iphoneos-arm.deb

3. Add a new repository to Cydia
In Cydia, go to Manage->Sources and Add the following repository:

http://leafmoon.users.sourceforge.net/cydia/

4. Install R the Statistical Language
Use Search to find it, if you cannot, make sure your user level is either Hacker or Developer level under Manage->Settings because R is a command-line program so it will not show up if you are at the User level. You can change your level back to User immediately afterwards.

5. Run R from command line
Load up MobileTerminal and run R by simply typing:

R

Other notes

  • As it is based on command-line, plots are generated as files in the directory you start R in, as opposed to being displayed on screen.
  • Packages need to be installed and updated with the install.packages() and upgrade.packages() commands respectively
  • I recommend portrait mode as the keyboard seems to take up less screen estate but that’s just personal preference.
  • I’ve only tried this on iPhone 4′s retina display and it is barely usable so perhaps older iPhones would struggle with the readability.
  • As of the time of writing this post, the packaged R version is R 2.10.1. If you need a newer version, you will have to wait for an updated package or compile from source.

Thanks to ech0chrome for the Cydia repository and the reference instructions.

Dreamweaver SFTP error – New SSH host key fix

7 Comments »

I use Dreamweaver (CS4) to maintain my personal homepage and synchronize my files to my department’s webserver over SFTP (FTP over SSH that is).  A few months ago, our server got updated and Dreamweaver could no longer connect via SFTP.  I was getting the following error:

An FTP error occured – cannot make connection to host.

I figured it that perhaps SFTP wasn’t running on the server or maybe the port had changed.  But a quick check through the sftp command line confirmed that indeed the process was still working.

It turns out that server reset had created a new SSH host key and the local fingerprint stored by Dreamweaver was not updated.  It’s strange because most other applications will inform you of the new host key and prompt you to overwrite the old fingerprint.  For Dreamweaver, you have to manually delete the ssh_hosts file in

~/Library/Application Support/Adobe/Dreamweaver CS4/en_US/Configuration

or at least delete the row corresponding to the server in question.

I also read this blog post that claims older versions of Dreamweaver had this support, but it was taken out for some mysterious reason.

How to install R packages from source on Mac OS X

3 Comments »

In my last post about installing the rgdal R package on Mac OS X, I had apparently glossed over the last step of compiling the R package from source.  I hadn’t realized that Mac OS X actually doesn’t come default with the necessary software to do this.  Most times, prepackaged binaries work fine but occasionally, packages may not have a binary available or may need to be customized.  For example, in rgdal, where the default search paths are incorrect for the GDAL frameworks on Mac OS X.

In order to compile a package from the source code, you’ll need the package source (typically a .tar.gz file on Mac OS X) and a compiler (e.g. gcc).  In my opinion, the easiest way to set up the necessary tools is to install Xcode from the Apple Developer site. You’ll need to register (free) for an Apple Developer Connection account to access the download.
(Update, May 20, 2011: Since version 4, Xcode is no longer free for non-developers but can be purchased for $4.99 from the Mac App Store. Xcode 3 remains free from the Apple Developer site and is sufficient if you just need the compilers installed).

Once Xcode has been installed, you’ll then have all the required tools to compile R packages from source.  To do so, simply navigate in Terminal to the folder you saved the source to (the .tar.gz file), and run:

R CMD INSTALL packagename.tar.gz

or some variety of the above with any custom configurations. The R command will automatically run the appropriate commands to compile the package. Happy compiling!

Installing rgdal on Mac OS X

19 Comments »

The rgdal package contains bindings from R to the GDAL (Geospatial Data Abstraction Library) as well as the PROJ library. It is a dependency for numerous spatial statistics packages. Note that the binaries on the rgdal homepage are outdated — the most up-to-date version of the rgdal package can be found on CRAN.

Unsurprisingly, rgdal requires both the GDAL and PROJ framework to run. Both frameworks for Mac OS X are kindly maintained by William Kyngesburye and can be found here:

Unix Compatibility Frameworks

I recommend installing GDAL Complete as it contains both required frameworks in a neat little package.
At the time of writing, there are two versions of GDAL Complete:

  • GDAL 1.7 Complete
  • GDAL 1.6 Complete

Both should work for rgdal but note that currently, the stable Mac OS X version of Quantum GIS (QGIS 1.4) only runs with GDAL 1.6, so if you need to use QGIS as well, I would recommend installing GDAL 1.6.  If you’ve installed GDAL 1.7, then you’ll have to use the developer’s release of QGIS.
(May 19, 2011 update: Install GDAL 1.8 Complete as it works for both standard and developer’s release of QGIS)

Next, assuming you installed GDAL above with the default paths, you’ll need to install the R package from source (get package source here) with modified arguments either with install.packages() or through the terminal:

R CMD INSTALL --configure-args='--with-gdal-config=/Library/Frameworks/GDAL.framework/unix/bin/gdal-config --with-proj-include=/Library/Frameworks/PROJ.framework/unix/include --with-proj-lib=/Library/Frameworks/PROJ.framework/unix/lib' rgdal_0.6-33.tar.gz

If you’re getting this error:

Error: package ‘rgdal’ is not installed for ‘arch=i386′

when trying to load the rgdal package, it is because your version of rgdal was not compiled for the 32-bit version of R. You’ll need to run the 64-bit version of R (R64.app on Macs).

You may also need to update the base version of the sp package as well as any other dependencies.

Hello world!

No Comments »

Throughout the course of my research, I come across a lot of obstacles and nuisances that are either undocumented or documented poorly.  Therefore, I decided to throw up this blog as a repository of sorts to store all of these little tidbits of information with the hope that it will save somebody somewhere a few minutes of frustration.  But even if it doesn’t, I’m sure it’ll help me in the future, if only to serve as a reference.

My research deals mainly with statistical computing and involves a lot of R programming so most of the posts will revolve around that.  I’m also an Apple user and although R on Macs has come a long way, there is still room for improvement.  Hopefully this blog will be a small step towards making Mac OS X a statistics research-friendly platform.