A Quick look at Flare and Python

I recently wrote a review of Flare Version 2 which is a recent extension to the Cresset portfolio with the introduction of Electrostatic Complementarity (EC), i.e. a comparison of electrostatics on both the small molecule ligand and the target protein DOI.

Electrostatic interactions between small molecules and their respective receptors are essential for molecular recognition and are also key contributors to the binding free energy. Assessing the electrostatic match of protein-ligand complexes therefore provides important insights into why ligands bind and what can be changed to improve binding. Ideally, ligand and protein electrostatic potentials at the protein-ligand interaction interface should maximize their complementarity while minimizing desolvation penalties.

In addition Flare version 2 includes a new Python API, that allows users to automate tasks by scripting, but also integration with other Python packages such as RDKit cheminformatics toolkit, Python modules for graphing, statistics (NumPy, SciPy, MatPlotLib), and Jupyter notebook integration, it is this aspect of Flare that is the subject of this review.

When you start Flare there is a Python tab, Python scripts can be run from the Flare GUI or by using the pyflare command line tool from the Terminal.

Cresset now have a GitLab repository that contains Python scripts used to extend and integrate Flare https://gitlab.com/cresset. This also includes some developer resources.

The easiest way to get the extensions is to download the whole repository by clicking the download button (which looks like a cloud with a down arrow) and selecting "Download zip". These can be unzipped and then installed with the installextensions.py script using the pyflare executable from the Terminal as shown below:

MacPro:flare-python-extensions-master Chris$ /Applications/Flare.app/Contents/MacOS/pyflare installextensions.py
Install directory [/Users/username/Library/Application Support/Cresset BMD/Flare/python/extensions]: 
Install Featured Extensions (y/n) [y]: 
Install Additional Extensions (y/n) [y]: 
Some extensions require 3rd party Python packages to be installed from the Python Package Index.
Download and install these packages now (y/n) [y]:

You will be prompted for an install location. Leave this blank to install to the default location (/Users/username/Library/Application Support/Cresset BMD/Flare/python/extensions). It should then look like this

If you want to write your own extensions there is a repository https://gitlab.com/cresset/flare-python-pyflare that contains a collection of Python scripts to access all the main Flare functions through the pyflare command line Python interpreter.

They can be run using

pyflare scriptname.py [options]

There are also a collection of Python example scripts to be used as templates by developers https://gitlab.com/cresset/flare-python-developer.

The Python window opens to reveal the Python Interpreter as shown in the image below. This is used for the majority of python scripting, however for simple one-line scripts there is also a Python console.

A simple Python script

This is just a simple script, there is a project open and this just illustrates a few features.

The script is shown in detail below.

The first part of the script gets the ligand as a SMILES string and prints it to the output pane. It then uses the SMILES string to add a new molecule to the ligand list. It is also possible to add a ligand by adding the SMILES as a plain text string.

When a ligand is added to the ligand table a number of properties are automatically generated, these can be accessed, for example to print the molecular weight.

print(ligand2.properties['MW'].value)

The values in the table can be modified, for example to change the name of the molecule

ligand2.properties['Title'].value = 'New Molecule'

We can also modify the ligand for example a quick minimisation, all the results of the script are shown in the image below.

Integration with Jupyter Notebook.

Project Jupyter exists to develop open-source software, open-standards, and services for interactive computing across dozens of programming languages. The Jupyter Notebook is rapidly becoming the standard notebook creating and sharing code and Python seems to be the lingua franca in cheminformatics and computational chemistry.

The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and more.

Integration with the Jupyter Notebook does require command-line access, but Cresset provided me with compressed archive (FlarePythonNotebook.zip) containing the required files and a detailed set of installation commands (these are for Mac OSX).

The Python Notebook extension is usually distributed through the GitLab Developer page https://gitlab.com/cresset/flare-python-developer and there are more detailed installation instructions there.

First download the archive (I created a folder called Flare), then navigate to the folder containing the downloaded archive, then unzip the archive:

MacPro:Flare Chris$ unzip FlarePythonNotebook.zip
    Archive:  FlarePythonNotebook.zip
   creating: FlarePythonNotebook/
  inflating: FlarePythonNotebook/greenlet-0.4.15-cp36-cp36m-macosx_10_8_x86_64.whl  
   creating: FlarePythonNotebook/pythonnotebook/
  inflating: FlarePythonNotebook/pythonnotebook/nblauncher.py  
  inflating: FlarePythonNotebook/pythonnotebook/htmlparsers.py  
  inflating: FlarePythonNotebook/pythonnotebook/myfiledialog.py  
  inflating: FlarePythonNotebook/pythonnotebook/__init__.py  
  inflating: FlarePythonNotebook/pythonnotebook/eventloop.py  
 extracting: FlarePythonNotebook/pythonnotebook/flare_logo_16x16.png  
 extracting: FlarePythonNotebook/pythonnotebook/notebook.png  
  inflating: FlarePythonNotebook/pythonnotebook/common.py  
  inflating: FlarePythonNotebook/pythonnotebook/configurelogging.py  
  inflating: FlarePythonNotebook/pythonnotebook/flarekernelmanager.py  
   creating: FlarePythonNotebook/pythonnotebook/templates/
  inflating: FlarePythonNotebook/pythonnotebook/templates/notebook.html  
  inflating: FlarePythonNotebook/pythonnotebook/flarenotebook.py  
  inflating: FlarePythonNotebook/pythonnotebook/directoryservice.py  
  inflating: FlarePythonNotebook/pythonnotebook/notebookpreload.py  
  inflating: FlarePythonNotebook/pythonnotebook/notebookdefaultscript.py  
  inflating: FlarePythonNotebook/ChEMBL_mols_in_Flare.ipynb  
MacPro:Flare Chris$

Now copy the PythonNotebook to the Cresset folder in Application Support.

MacPro:Flare Chris$ cp -R FlarePythonNotebook/pythonnotebook ~/Library/Application\ Support/Cresset\ BMD/Flare/python/extensions
MacPro:Flare Chris$

Then use PIP to install a number of Python components into the Flare Python environment.

MacPro:Flare Chris$ /Applications/Flare.app/Contents/MacOS/pyflare -m pip install --user FlarePythonNotebook/greenlet-0.4.15-cp36-cp36m-macosx_10_8_x86_64.whl
Processing ./FlarePythonNotebook/greenlet-0.4.15-cp36-cp36m-macosx_10_8_x86_64.whl
Installing collected packages: greenlet
Successfully installed greenlet-0.4.15

The greenlet package supports lightweight concurrent programming, this is is actually a dependency of chemblwebresourceclient, which is a module used by the ChEMBLmolsin_Flare.ipynb example notebook.

A “greenlet”, is a primitive notion of micro-thread with no implicit scheduling; coroutines, in other words. This is useful when you want to control exactly when your code runs.

MacPro:Flare Chris$ /Applications/Flare.app/Contents/MacOS/pyflare -m pip install --user jupyter pillow chembl_webresource_client

This will install various Python packages and their dependencies.

Successfully installed Send2Trash-1.5.0 appnope-0.1.0 asn1crypto-0.24.0 backcall-0.1.0 bleach-3.1.0 cffi-1.12.2 chembl-webresource-client-0.9.31 cryptography-2.6.1 decorator-4.4.0 defusedxml-0.5.0 easydict-1.9 entrypoints-0.3 gevent-1.4.0 gevent-openssl-1.2 grequests-0.2.0 ipykernel-5.1.0 ipython-7.3.0 ipython-genutils-0.2.0 ipywidgets-7.4.2 jedi-0.13.3 jsonschema-3.0.1 jupyter-1.0.0 jupyter-client-5.2.4 jupyter-console-6.0.0 jupyter-core-4.4.0 mistune-0.8.4 nbconvert-5.4.1 nbformat-4.4.0 notebook-5.7.6 pandocfilters-1.4.2 parso-0.3.4 pexpect-4.6.0 pickleshare-0.7.5 pillow-5.4.1 prometheus-client-0.6.0 prompt-toolkit-2.0.9 ptyprocess-0.6.0 pyOpenSSL-19.0.0 pycparser-2.19 pyrsistent-0.14.11 pyzmq-18.0.1 qtconsole-4.4.3 requests-cache-0.4.13 terminado-0.8.1 testpath-0.4.2 tornado-6.0.1 traitlets-4.3.2 unittest2py3k-0.5.1 unittest2six-0.0.0 wcwidth-0.1.7 webencodings-0.5.1 widgetsnbextension-3.4.2

As time goes by you may update the Flare Python environment using

/Applications/Flare.app/Contents/MacOS/pyflare -m pip install --user <name of Python packages you want to update/install>

Once done you will probably get a series of warnings like this.

 The script jupyter-trust is installed in '/Users/Chris/Library/Application Support/Cresset BMD/Flare/python/user-site-packages/bin' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

Now when we start Flare the Python tab gives access to the Jupyter notebook

Clicking on the Python Notebook button opens up a new Jupyter Notebook.

One thing to note is that the notebook is created in a Temp folder. This means that when you open an existing notebook a copy is created in a temp folder, a consequence is that after editing the notebook clicking the save notebook icon in the Jupyter notebook does not update the copy loaded. In addition any files created images, exports etc. are by default saved in the temp folder, not the folder where the original notebook was loaded from. To Load/Save/Save As you will need to use the buttons at the bottom right of the notebook window. The same is true for kernel-releated operations (Restart, Clear cells, etc.), which are associated to the bottom left corner.

Inside the Jupyter notebook the flare_ipynb_dir variable gives you the full path to where temporary files are saved, e.g.

/var/folders/zq/_y_m2txn6lx1dc1by89jd_xc0000gn/T/flare_ipynb.1197

If you attempt to close the notebook or Flare while the current work is not saved you will be asked if you are sure you want to leave without saving your work.

Cresset provided me with a demo notebook to try out. Rather than going through every cell in detail I thought it might be better to just show a video of it in action. To try it yourself, click on the Python Notebook button then loading the notebook.

Click here to Open in your web browser

As you can see the integration between Flare and RDKit works really nicely and the notebook can be used to provide a record the whole session for sharing (remember to save!).

Accessing External Python Machine Learning Models

Using a template notebook provided by Cresset I looked at accessing an external Random Forest model I had created to predict AMES activity using a python script I had written to access the model from the command line. This script simply requires a SMILES string as input and returns the result Positive/Negative

MacPro:~ $ python /Users/username/Projects/AMES_data/AMESTest/AMESmachineLearningModelOnlySMILESinput.py "[H]OS(=O)C1=C([H])C([H])=C(Cl)C([H])=C1[H]"
Negative

The key cells in the notebook are the definition of the call to the external script, you will need replace /Users/username/Projects/AMES_data/AMESTest/AMESmachineLearningModelOnlySMILESinput.py with the absolute path to your script, it is probably a good idea to also include the path to the Python used to create the model.

def call_external_script(smiles):
    PYTHON_SCRIPT = "/Users/username/Projects/AMES_data/AMESTest/AMESmachineLearningModelOnlySMILESinput.py"
    PYTHON_EXE = "/Users/username/miniconda3/bin/python"
    cmdline = [PYTHON_EXE, PYTHON_SCRIPT, smiles]
    proc = subprocess.Popen(cmdline, stdin=subprocess.PIPE,
        stdout=subprocess.PIPE, stderr=subprocess.PIPE,
        startupinfo=StartupInfo().get())
    out, err = proc.communicate()
    if (out is None):
        out = ""
   if err:
        sys.stderr.write("ERROR: {0:s}\n".format(err.decode("UTF-8")))
    return out.decode("UTF-8").strip()

We also need a cell to provide callbacks which are called when ligands are loaded into the ligand table in Flare

flare.callbacks.ligands_added.add(compute_mlmodel)
flare.callbacks.ligand_structure_changed.add(compute_mlmodel)
flare.callbacks.poses_added.add(compute_mlmodel)
flare.callbacks.pose_structure_changed.add(compute_mlmodel)

You can download the Jupyter Notebook here

The first time I tried this I got an error

  File "src/gevent/libev/corecext.pyx", line 669, in gevent.libev.corecext.loop.child
  File "src/gevent/libev/corecext.pyx", line 1190, in gevent.libev.corecext.child.__cinit__
TypeError: child watchers are only available on the default loop

A search on Stackoverflow and a chat with Cresset support identified gevent as the problem.

The gevent monkey patch is required by the chembl_webresource_client module. This is the module provided by ChEMBL to access their web API, and it requires monkey-patching the Python instance to work, which may cause problems elsewhere as several standard Python modules (including subprocess) are replaced by gevent-provided modules. .

The solution is to clear the notebookpreload.py script:

I did this by changing the name of the file /Users/Chris/Library/Application Support/Cresset BMD/Flare/python/extensions/pythonnotebook/notebookpreload.py to "notebookpreload_chembl.py and creating a blank file called notebookpreload.py as shown in the Finder window below.

I then restarted Flare, reloaded the notebook into the Flare Jupyter Notebook and ran all the cells. I then loaded a set of ligands and it all worked perfectly as shown below, the column "MLModel" contains the AMES prediction result.

Summary

In conclusion, the introduction of the Python interface dramatically increase the capabilities of Flare and allows it to be integrated into a variety of existing tools, models and workflows. This is a very new extension to Flare and it still feels a little experimental, there is little written help and few examples available to download, however the Cresset support is absolutely fantastic. At the moment I'd say it is the domain of the expert coder, I'm sure as a few things get ironed out it will be become an essential part of Flare.

I just found out that Cresset have their own YouTube channel and it looks like they will be using it to demo features in their software https://www.youtube.com/channel/UC2p8MEXqbD509z6VXujgg.

Last updated 9 April 2019