FloatingPointError: divide by zero encountered in true_divide

[SternLabTAU]

When running the Exp_CM model, the error below keeps coming up. This occurs even when running on the beta_lactamase dataset given as an example:

module load python/python-anaconda3.2019.7
phydms betaLactamase_alignment.fasta.phy.fasta betaLactamase_alignment.fasta.phy_phyml_tree.txt ExpCM_betaLactamase_prefs.txt test.out
/powerapps/share/python-anaconda-3.2019.7/lib/python3.7/site-packages/Bio/Alphabet/__init__.py:26: PendingDeprecationWarning: We intend to remove or replace Bio.Alphabet in 2020, ideally avoid using it explicitly in your code. Please get in touch if you will be adversely affected by this. https://github.com/biopython/biopython/issues/2046
  PendingDeprecationWarning)
2020-01-09 16:57:12,435 - INFO - Beginning execution of phydms in directory /sternadi/home/volume1/sternadi/CVB3_RG/test

2020-01-09 16:57:12,436 - INFO - Progress is being logged to test.out_log.log

2020-01-09 16:57:12,436 - INFO - Version information:
	Time and date: Thu Jan  9 16:57:08 2020
	Platform: Linux-3.10.0-514.el7.x86_64-x86_64-with-centos-7.3.1611-Core
	Python version: 3.7.3 (default, Mar 27 2019, 22:11:17)  [GCC 7.3.0]
	phydms version: 2.3.6
	Bio version: 1.74
	cython version: 0.29.12
	numpy version: 1.16.4
	scipy version: 1.3.0
	matplotlib version: 3.1.0
	natsort version: 6.0.0
	sympy version: 1.4
	six version: 1.12.0
	pandas version: 0.24.2
	pyvolve version: 1.0.0
	statsmodels version: 0.10.0
	weblogolib version: 3.5.0
	PyPDF2 version: 1.26.0

2020-01-09 16:57:12,437 - INFO - Parsed the following arguments:
	alignment = betaLactamase_alignment.fasta.phy.fasta
	tree = betaLactamase_alignment.fasta.phy_phyml_tree.txt
	model = ('ExpCM', 'betaLactamase_prefs.txt')
	outprefix = test.out
	brlen = optimize
	gammaomega = False
	gammabeta = False
	omegabysite = False
	omegabysite_fixsyn = False
	diffprefsbysite = False
	diffprefsprior = ('invquadratic', 150.0, 0.5)
	fitphi = False
	randprefs = False
	avgprefs = False
	divpressure = None
	ncpus = 1
	fitprefsmethod = 2
	ncats = 4
	minbrlen = 1e-06
	minpref = 0.002
	seed = 1
	initparams = None
	profile = False
	opt_details = False
	nograd = False

2020-01-09 16:57:12,437 - INFO - Random number seed: 1

2020-01-09 16:57:12,438 - INFO - Reading alignment from betaLactamase_alignment.fasta.phy.fasta
2020-01-09 16:57:13,083 - INFO - Read 50 aligned sequences from betaLactamase_alignment.fasta.phy.fasta, each consisting of 263 codons.

2020-01-09 16:57:13,084 - INFO - The model will be an ExpCM informed by site-specific amino-acid preferences in betaLactamase_prefs.txt
2020-01-09 16:57:14,735 - INFO - Successfully read site-specific amino-acid preferences for all 263 sites.

2020-01-09 16:57:14,736 - INFO - Nucleotide frequency parameters phi will be set so stationary state matches alignment nucleotide frequencies of A = 0.239, C = 0.269, G = 0.292, T = 0.201

2020-01-09 16:57:15,055 - ERROR - Terminating phydms at Thu Jan  9 16:57:15 2020 with ERROR
Traceback (most recent call last):
  File "/powerapps/share/python-anaconda-3.2019.7/bin/phydms", line 299, in main
    freeparams=freeparams)
  File "/powerapps/share/python-anaconda-3.2019.7/lib/python3.7/site-packages/phydmslib/models.py", line 1279, in __init__
    omega=omega, beta=beta, mu=mu, freeparams=freeparams)
  File "/powerapps/share/python-anaconda-3.2019.7/lib/python3.7/site-packages/phydmslib/models.py", line 434, in __init__
    self.updateParams({}, update_all=True)
  File "/powerapps/share/python-anaconda-3.2019.7/lib/python3.7/site-packages/phydmslib/models.py", line 539, in updateParams
    self._update_Frxy()
  File "/powerapps/share/python-anaconda-3.2019.7/lib/python3.7/site-packages/phydmslib/models.py", line 706, in _update_Frxy
    / (1 - self.piAx_piAy_beta), where=scipy.logical_and(
FloatingPointError: divide by zero encountered in true_divide
Traceback (most recent call last):
  File "/powerapps/share/python-anaconda-3.2019.7/bin/phydms", line 551, in <module>
    main() # run the script
  File "/powerapps/share/python-anaconda-3.2019.7/bin/phydms", line 299, in main
    freeparams=freeparams)
  File "/powerapps/share/python-anaconda-3.2019.7/lib/python3.7/site-packages/phydmslib/models.py", line 1279, in __init__
    omega=omega, beta=beta, mu=mu, freeparams=freeparams)
  File "/powerapps/share/python-anaconda-3.2019.7/lib/python3.7/site-packages/phydmslib/models.py", line 434, in __init__
    self.updateParams({}, update_all=True)
  File "/powerapps/share/python-anaconda-3.2019.7/lib/python3.7/site-packages/phydmslib/models.py", line 539, in updateParams
    self._update_Frxy()
  File "/powerapps/share/python-anaconda-3.2019.7/lib/python3.7/site-packages/phydmslib/models.py", line 706, in _update_Frxy
    / (1 - self.piAx_piAy_beta), where=scipy.logical_and(
FloatingPointError: divide by zero encountered in true_divide

[jbloom]

Can you post a ZIP file with the exact inputs you are using to run your command of:

      phydms betaLactamase_alignment.fasta.phy.fasta betaLactamase_alignment.fasta.phy_phyml_tree.txt ExpCM_betaLactamase_prefs.txt test.out

I guess these would be the file betaLactamase_alignment.fasta.phy.fasta, betaLactamase_alignment.fasta.phy_phyml_tree.txt, and betaLactamase_alignment.fasta.phy_phyml_tree.txt

I think some but not all of these files are in the tutorial, so I just want to make sure I am debugging with the correct input files.

[SternLabTAU]

Package and support much appreciated!! Attached are the files used for input ...
Archive.zip

[jbloom]

@SternLabTAU: Are you running phydms on a Windows machine?

I don't get the error on Linux or Mac, but Googling around it sounds like there can be some differences in floating point match on Windows versus Linux / Mac that could be causing small numbers to be rounded to zero.

It seems like there are various compiler flags that can be set to fix this, but since I can't reproduce the error myself on my computer it's hard for me to try these out.

So basically:

1. Can you let me know what computer you are using?

2. If it is Windows, do you have the option of trying Mac or Linux? The installation docs technically say phydms is only tested on Mac or Linux.

3. If Windows isn't causing the issue, or if you don't have a Mac or Linux option, I will look into this more.

[SternLabTAU]

I ran phydms on Linux with CentOS7. I can try and install on a Mac and see if the error is reproduced.

[jbloom]

OK, let me know if that works. Sorry...

[SternLabTAU]

Same error on a Mac:

2020-01-15 14:14:04,957 - INFO - Progress is being logged to test_log.log

2020-01-15 14:14:04,957 - INFO - Version information:
	Time and date: Wed Jan 15 14:14:03 2020
	Platform: Darwin-18.7.0-x86_64-i386-64bit
	Python version: 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 13:42:17)  [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]
	phydms version: 2.3.8
	Bio version: 1.76
	cython version: 0.29.13
	numpy version: 1.17.2
	scipy version: 1.3.1
	matplotlib version: 3.1.1
	natsort version: 7.0.0
	sympy version: 1.4
	six version: 1.12.0
	pandas version: 0.25.1
	pyvolve version: 1.0.0
	statsmodels version: 0.10.1
	weblogolib version: 3.5.0
	PyPDF2 version: 1.26.0

2020-01-15 14:14:04,957 - INFO - Parsed the following arguments:
	alignment = betaLactamase_alignment.fasta.phy.fasta
	tree = betaLactamase_alignment.fasta.phy_phyml_tree.txt
	model = ('ExpCM', 'betaLactamase_prefs.txt')
	outprefix = test
	brlen = optimize
	gammaomega = False
	gammabeta = False
	omegabysite = False
	omegabysite_fixsyn = False
	diffprefsbysite = False
	diffprefsprior = ('invquadratic', 150.0, 0.5)
	fitphi = False
	randprefs = False
	avgprefs = False
	divpressure = None
	ncpus = 1
	fitprefsmethod = 2
	ncats = 4
	minbrlen = 1e-06
	minpref = 0.002
	seed = 1
	initparams = None
	profile = False
	opt_details = False
	nograd = False

2020-01-15 14:14:04,957 - INFO - Random number seed: 1

2020-01-15 14:14:04,957 - INFO - Reading alignment from betaLactamase_alignment.fasta.phy.fasta
2020-01-15 14:14:05,180 - INFO - Read 50 aligned sequences from betaLactamase_alignment.fasta.phy.fasta, each consisting of 263 codons.

2020-01-15 14:14:05,180 - INFO - The model will be an ExpCM informed by site-specific amino-acid preferences in betaLactamase_prefs.txt
2020-01-15 14:14:05,576 - INFO - Successfully read site-specific amino-acid preferences for all 263 sites.

2020-01-15 14:14:05,576 - INFO - Nucleotide frequency parameters phi will be set so stationary state matches alignment nucleotide frequencies of A = 0.239, C = 0.269, G = 0.292, T = 0.201

2020-01-15 14:14:05,709 - ERROR - Terminating phydms at Wed Jan 15 14:14:05 2020 with ERROR
Traceback (most recent call last):
  File "/Users/adi/anaconda/bin/phydms", line 300, in main
    freeparams=freeparams)
  File "/Users/adi/anaconda/lib/python3.6/site-packages/phydmslib/models.py", line 1283, in __init__
    omega=omega, beta=beta, mu=mu, freeparams=freeparams)
  File "/Users/adi/anaconda/lib/python3.6/site-packages/phydmslib/models.py", line 438, in __init__
    self.updateParams({}, update_all=True)
  File "/Users/adi/anaconda/lib/python3.6/site-packages/phydmslib/models.py", line 543, in updateParams
    self._update_Frxy()
  File "/Users/adi/anaconda/lib/python3.6/site-packages/phydmslib/models.py", line 710, in _update_Frxy
    / (1 - self.piAx_piAy_beta), where=numpy.logical_and(
FloatingPointError: divide by zero encountered in true_divide
Traceback (most recent call last):
  File "/Users/adi/anaconda/bin/phydms", line 552, in <module>
    main() # run the script
  File "/Users/adi/anaconda/bin/phydms", line 300, in main
    freeparams=freeparams)
  File "/Users/adi/anaconda/lib/python3.6/site-packages/phydmslib/models.py", line 1283, in __init__
    omega=omega, beta=beta, mu=mu, freeparams=freeparams)
  File "/Users/adi/anaconda/lib/python3.6/site-packages/phydmslib/models.py", line 438, in __init__
    self.updateParams({}, update_all=True)
  File "/Users/adi/anaconda/lib/python3.6/site-packages/phydmslib/models.py", line 543, in updateParams
    self._update_Frxy()
  File "/Users/adi/anaconda/lib/python3.6/site-packages/phydmslib/models.py", line 710, in _update_Frxy
    / (1 - self.piAx_piAy_beta), where=numpy.logical_and(
FloatingPointError: divide by zero encountered in true_divide

[jbloom]

I'm not sure why you are getting these errors on your installations and we aren't. It's obviously hard to debug because we can't reproduce it. But I wonder if it has to do with something else in your anaconda environment?

@skhilton: Would you want to make a full conda package for phydms, and we can then make sure that works for us and let @SternLabTAU try installing that? That way we can ensure that most of the other libraries they are using are the same as ours. I suspect the problem has to do with some difference in numpy / scipy across the installations right now.

[skhilton]

@jbloom: sounds good. I will work on that today.

[skhilton]

I was not able to reproduce the error on my local or on the server. However, I was able to reproduce the error using this conda environment. I am not exactly sure what the difference is but I can start debugging using this environment.
phydms_env.txt

[skhilton]

@SternLabTAU: Sorry for this error! We think we've come up with a partial solution. We think this error is a results of some scipy or numpy installation error. We think that installing scipy and numpy using pip within a conda(https://docs.conda.io/en/latest/miniconda.html) environment should solve the issue.

I've attached a directory with 5 files:
1-3: your data files (preferences, tree, fasta)
4: a conda environment file which loads python, pip, and then installs phydms
5: a small bash script to run the phydms command.

I would suggest:

1. running conda env create -f phydms_env.yml
2. running conda activate phydms_env
3. which phydms The phydms should be in the conda environment and not your .local bin
4. running bash run_phydms.bash
5. running conda deactivate

Please let us know by posting in this thread if this solution works or doesn't work or if you have any questions!

floating_point_error_phydms.zip

[SternLabTAU]

thanks, will give it a shot asap!