Installing scientific Python on Mac OS X

Getting scientific Python running on a Mac is one of the biggest hurdles for data scientists who are just getting started (and, trust us, professionals too!). We'd usually steer readers toward one of the more popular articles on the subject, but it's gotten a bit stale. Therefore, here are updated step-by-step instructions for getting a basic environment set up.

In this guide we'll install the following packages:

  1. Quickstart
  2. Homebrew
  3. Python
  4. virtualenv and virtualenvwrapper
  5. NumPy
  6. SciPy
  7. matplotlib
  8. IPython and the Qt console

 

Quickstart

For the impatient, the following code will get you up and running without delay. Otherwise, please continue reading the rest of this post for step-by-step instructions and explanations.

NB: This code assumes Homebrew is installed; if it isn't, follow the instructions below! Also, be sure to follow any instructions Homebrew gives after each installation, such as modifying your path variables.

 

Homebrew

Homebrew's homepage puts it well: "Homebrew installs the stuff you need that Apple didn’t." (It used to say, "Homebrew is the easiest and most flexible way to install the UNIX tools Apple didn't include with OS X," which may not be as catchy as the new wording, but does a better job explaining.) You will need to have XCode and its Command Line Tools installed first (get them in the App Store). Then, install Homebrew on your machine by pasting this into Terminal:

That's it. Try running brew update and brew doctor at the command line to make sure it's working properly.

You'll need to add the Homebrew directory to the front of your system path, in order to ensure that Homebrew-installed software is given priority over any other versions. To do so, open the .bash_profile in your user directory (that's the ~ directory or, more verbosely, /Users/[your_user_name]) and add the following line (if you don't have a .bash_profile, just create one with any text editor):

Restart Terminal so it picks up the new path. Homebrew is generally very good about alerting you to any action you need to take after it runs, including path modifications. Make sure to pay attention to its output.

 

Python

Now to the main event: Python. These instructions are for version 2.7.3. If you want to install something else, like Python3, just adjust the commands as required. With Homebrew, getting Python is quite easy:

This will also install package management tools like pip, which we'll need later. To get them to work seamlessly, add this to your .bash_profile:

To verify the installation, type which python into Terminal. You should see /usr/local/bin/python in response.

 

virtualenv

virtualenv is a tool that allows you to create isolated Python environments, each with its own set of packages and dependencies. This is useful for testing or managing package requirements (for example, if you build an application that is dependent on a certain version of a third-party package but another application requires a more recent version, you might break the first application by upgrading). This is not required, and all the commands below should work whether or not you are using virtualenv, so consider this step for convenience only. The only difference will be that directories (such as that returned by which pip) will point to the virtualenv rather than /usr/local.

First, install virtualenv and virtualenvwrapper, a tool that makes working with virtualenv somewhat easier:

Next, source the virtualenvwrapper script:

This will create a (hidden) virtualenv directory at ~/.virtualenv. Now you can create your first virtual environment:

Your new virtualenv test1 comes with a complete install of Python 2.7.3 and its own version of pip. It is activated by default, so running any pip command will only impact this environment. Note that if you deactivate the virtualenv, you will lose access to any packages installed in it. You can switch between virtualenvs with the workon command. To delete your test virtualenv, run rmvirtualenv test1.

 

Numpy

NumPy, of course, forms the basis of most scientific work in Python. NumPy can be installed with pip, but that method failed on Macs until quite recently (see pip issue #707). If you'd like to try it, simply execute the following in Terminal:

 

SciPy

SciPy requires a Fortran compiler. To acquire one, use Homebrew:

Once that installs, you may choose to install SciPy with pip:

 

matplotlib

Installing matplotlib, a versatile plotting library, follows the same pattern. However, you may have to install freetype first:

Either use pip:

 

IPython

If you're not using IPython as your interactive console, you should be. It improves on the stock Python console in every way. Installing IPython itself is a fairly straightforward pip command:

Getting the Qt Console to run is more difficult, but well worth it. First, you'll need to download the Qt library from this site. Once you have that set up, begin installing the prerequisites:

After installing pyqt, Homebrew will prompt you to add the following to your .bash_profile:

Continue installing prerequisites:

And that's all -- you should be able to launch the Qt console by executing ipython qtconsole. If you want to see the matplotlib output inline (and why wouldn't you?), then execute:

Fin.

We hope these instructions remove a common stumbling block for any data scientists getting started with scientific Python... and also those professionals who need a checklist for setting up new environments! After all, at some point we all start from square one.

33 Responses to “Installing scientific Python on Mac OS X”

  1. Why does the old page that links here ( http://www.thisisthegreenroom.com/2011/installing-python-numpy-scipy-matplotlib-and-ipython-on-lion/#python ) say that python should be built with --framework and --universal but those flags are missing here? Are they the default now?

    Reply

    • That's correct: the --universal flag is no longer needed due to improvements in Homebrew and the --framework flag is now enabled by default.

      Reply

  2. Hi,

    I'm following this guide, and I found something that worked for me different that what you write. Installing python, and setting the correct value in .bash_profile on mountain lion. my line looks like

    export PATH=/usr/local/bin:$PATH

    Thanks for the guide!

    Reply

  3. Hi -
    Thanks for the great writeup -- it works really well on Mountain Lion (ver. 10.8.3). I did have to execute this after getting to the end and updating the $PYTHONPATH:
    easy_install readline
    pip install matplotlib
    I'm not sure why they weren't already installed following your steps. In any case, executing those commands got everything in order so I could:
    ipython qtconsole --pylab=inline
    without errors.

    Reply

  4. I had to add:
    brew install libpng

    before I could get matplotlib to compile. If you are getting an error that png is missing, give that a try. This is on mountain lion. hope that helps someone :)

    Reply

  5. There is also an active solution, hosted on github, by Fonnesbeck:
    https://github.com/fonnesbeck/ScipySuperpack

    Reply

  6. [...] several Macbook systems for scientific Python and there were enough gotchas to write this up. The excellent install post at Lowin Data Company is the latest successor to ‘This is the Green Room‘ for Mac Lion. [...]

    Reply

  7. If you type in: which python
    and get something other than the path you said we should
    (i.e., /Library/Frameworks/Python.framework/Versions/2.7/bin/python)

    what is wrong and how do we fix it?

    Reply

    • Hi Mike, you need to check your PATH settings -- most likely it hasn't been updated to reflect the new software. The system will search the PATH in order, accessing the first Python it comes across (reported by "which python"), so you'll need to make sure the new Python appears earlier on the PATH.

      Reply

  8. Question: Why do you install numPy, isn't it included in SciPy anyway?

    Reply

  9. There are also scientific computing Python distros out there for free, like Enthought Canopy, Express version: https://www.enthought.com/products/canopy/

    Reply

  10. [...] to get started with some coding to apply the lessons you have learnt. After much trial and error, this post by the Lowin Data Company came to my rescue. It details installing a Python environment on a Mac OS [...]

    Reply

  11. Hi,

    I want to start installing everything python related from scratch as it's described on this page, because I've had some problems from trying to install python related packages from different sources in the past. How do I find out all of the places that python is located and remove all of them so that I can be sure I'm starting from scratch?

    Thanks,
    -Adam

    P.S. I tried your prescription and it didn't work, I believe that the problem is in that I have most of these things downloaded already, so instead of re-installing it in the proper place, it just tells me that it's already installed.

    Reply

  12. Hi thanks for you tutorial, i think is the best of the web.
    But I have a problem it is two days and i cannot figure out. I have maverick but after I installed everything python told me when i tried to import matplotlib.pyplot as plt.
    Traceback (most recent call last):
    File "", line 1, in
    AttributeError: 'module' object has no attribute 'pyplot'
    I really do not know. I found only during the matplotlib installation this problem in the REQUIRED DEPENDENCIES AND EXTENSIONS check:
    pycxx: yes [Couldn't import. Using local copy.]
    libagg: yes [pkg-config information for 'libagg' could not
    be found. Using local copy.]

    Reply

  13. Thank you! I was off in the weeds somewhere trying to get this working before I found your guide.

    Reply

  14. Hi, your information is absolutely useful to the person who is not familiar with computer commands like me. Thank you for your post. I have a question. When I run 'which python', the result path looks '/usr/bin/python', which is not the path you mentioned. Is it ok? Or how can I change it?
    FYI, I just bought a new macbook. There was nothing previously installed on my system. Everything has been done is what you write in this post. thank you

    Reply

  15. Hi Mike,

    I am attempting to install the scipy stack on my new mac which was shipped with OS Mavericks. My problem may not be related to mavericks though. I have homebrew installed and working. I begin with a clean bill of health (brew doctor reports I am good to go). I then proceed as you describe above, except without the virtualenv (I am not opposed, I have just never done that and am attempting to only change one new thing at a time). Here is what I have, and where it crashes

    brew install python
    pip install numpy [I verified pip is from my brewed python - this works]
    pip install scipy [this works]
    pip intsall matplotlib - fails

    after checking a working backend, it fails with the terminal message
    "Fatal Python error: PyThreadState_Get: no current thread"
    The long problem report includes what I suspect are likely the problem (along with MANY more lines), things like
    "Binary Images:
    ...
    0x7fff8d613000 - 0x7fff8d781ff7 libBLAS.dylib (1094.5) /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
    ...
    "
    so I can see that the BLAS dynamic library is linked against the /System/Library python and not my brewed python :(
    I read a tool "install_name_tool" might be able to fix my problem, but I have never used that, and fear breaking something.

    Have you seen a problem like this? Would you know how to fix? A specific example how to use install_name_tool would be fantastic. However, I suspect I do not want to move the dylib link, but somehow copy, since my /System/Library/...python shipped with BLAS and LAPACK...

    Also - I have posted this on stack overflow:
    http://stackoverflow.com/questions/20167864/pip-install-matplotlib-fails-on-mavericks-with-homebrew

    Any help would be appreciated. Later, I will attempt to do the same, except inside a virtualenv, just to see if that helps, but looking at your instructions above, I don't see that it will help, as you install matplotlib before creating a virtualenv.

    Thanks,

    Andre

    Reply

  16. Hi,

    I am trying this on OS X Mavericks.

    When I try to source virtualenv using
    'source /usr/local/share/python/virtualenvwrapper.sh'

    I get an error: "-bash: /usr/local/share/python/virtualenvwrapper.sh: No such file or directory"

    Do you know how to correct this?

    Reply

  17. [...] from the ground up, and I’ve decided to try out a brewed Python. I’ve been following these very elegant instructions to set up numpy, scipy, etc, on OSX, and it worked flawlessly! Until [...]

    Reply

  18. [...] Summary ・Pythonのnumpy, scipy, matplotlib等のinstallははまるとミスりやすいのでやり方をまとめる。 ・以下のPageのやり方が一番良い(気がする)のでまとめる ・・http://www.lowindata.com/2013/installing-scientific-python-on-mac-os-x/ [...]

    Reply

  19. Hello

    and thank you for this simple guide. It would have spared me a complete OS wipe if I had found the guide earlier. Though I managed to mess sth up again. When I type "which -a python" or "which -a pip" to check for the locations, I get /usr/local/bin/python (or /pip) twice (and of course /usr/bin/python too, but this entry appears only ones). I double checked the .bash_profile, it contains only the two statements from this guide. Since I followed this guide starting with a nice new clean OSx, there should have been no previous installation of python via homebrew. Everything still works fine, but I wonder where the doubled entry comes from ...

    Reply

  20. This was very helpful. Thanks!

    I had a few issues:

    instead of
    source /usr/local/share/python/virtualenvwrapper.sh
    I needed
    source /usr/local/bin/virtualenvwrapper.sh

    I needed to do this before installiong matplotlib
    cd /usr/local/include; ln -s freetype2 freetype

    brew install pyqt
    produces the message 'Phonon support is broken.'

    Instead of
    export PYTHONPATH=/usr/local/lib/python:$PYTHONPATH
    I needed
    export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH

    Reply

  21. I try it on mavericks.But failed while installing matplotlib and compiler says "fatal error: 'freetype/config/ftheader.h' file not found"

    Make a symbol link fix this:
    ln -s /usr/local/include/freetype2 /usr/local/include/freetype

    seems an update of freetype have broken this dependency.

    Reply

  22. Thanks for the walk-through. It was very helpful and much appreciated.

    Reply

  23. [...] do this properly (i.e. install Python and the libraries somewhat manually). I found this website (http://www.lowindata.com/2013/installing-scientific-python-on-mac-os-x/) to be a real help for a middling computer geek like myself (know what PATH is but pretty scared [...]

    Reply

  24. I found that I had to modify your quickstart instructions a bit. clang is quite strict, so I had to do

    for numpy and matplotlib, and ipython

    I also had to symlink in the freetype headers

    Thanks.

    Reply

  25. Excellent. Very helpful.

    You might want to note that virtualenvwrapper.sh may end up in a different place (/usr/local/bin/virtualenvwrapper.sh on my system), and that the line that sources it should be added to your .bash_rc/.profile.

    Reply

  26. Also note that matplotlib has an X windows dependency. On later versions of OS X that do not have X11 built in, you will have to install XQuartz from a .DMG file on http://xquartz.macosforge.org/.

    Reply

  27. [...] Basic Packages Python, virtualenv, NumPy, SciPy, matplotlib and IPython [...]

    Reply

  28. Ok so i went through all of the steps and my terminal says it is all installed but then where do i go to find the program? this might be a stupid question but i just don't know where to go. where is python? haha

    Reply

  29. In recent Homebrew, brew install gfortran barfs. Need to run brew install gcc instead.

    Reply

Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">