Possble improvements in segregation outputs

[XingerTang]

AlphaImpute2 version: v0.0.2

Current segregation outputs are based on the function Heuristic_Peeling.fillPointEstimates, which only uses known haplotypes to estimate the segregation probabilities. This ensures the quality of the data, such that we transmit the genotype probabilities between the parents and children only when we are confident in how the haplotypes are inherited.

But if we are using this for the outputs, we are holding back a lot of information which we are possibly able to obtain.
For the accuracy test case with 5 generations of input data, AlphaPeel is able to give the segregation probabilities of:

• Generation 0 - 1
• Generation 1 - 2
• Generation 2 - 3
• Generation 3 - 4

In total, 4 sets of data.

While the restriction of AlphaImpute2 leads to segregation probabilities of:

• Generation 1 - 2
• Generation 2 - 3
• Generation 3 - 4

In total, 3 sets of data, which is 1 less than the AlphaPeel.

Proposed fix:

• Write a new function that generates segregation probabilities based on genotype probabilities instead of haplotypes to produce the output segregation probabilities.

[gregorgorjanc]

@XingerTang Interesting nut to crack or shall we simply avoid it!? Ok, let's say we get all the information between all the generations, even if very diffuse, how will we use it down the line? I agree that completeness is always useful, so from that perspective I vote that we would cover all sequential pairs of generations. But, if most of these seg prob are diffuse, will we be able to use them at all in the downstream work? I guess this last question is the real nut to crack? Shall we discuss in person and then decide how to handle it?

[XingerTang]

@gregorgorjanc

The segregation probabilities would affect the recombination prediction or the quality of the generated ARG.

The following are the AlphaPeel accuracy test results:

---------------------------------------------------------- seg_prob ----------------------------------------------------------
Test Num             Population Accu      Gen1 Accu            Gen2 Accu            Gen3 Accu            Gen4 Accu            Gen5 Accu            Test Name                           
3                    0.743                nan                  0.105                0.942                0.957                0.940                multi                               
4                    0.743                nan                  0.106                0.942                0.957                0.940                multi_est_alt_allele_prob           
5                    0.740                nan                  0.109                0.929                0.954                0.939                multi_est_alt_allele_prob_est_geno_error_prob_est_seq_error_prob 
6                    0.706                nan                  -0.008               0.885                0.953                0.964                multi_seq_file                      
7                    0.716                nan                  0.013                0.897                0.955                0.963                multi_est_alt_allele_prob_seq_file  
8                    0.716                nan                  0.013                0.897                0.956                0.964                multi_est_alt_allele_prob_est_geno_error_prob_est_seq_error_prob_seq_file 

For AlphaImpute2, with the same test and the same dataset:

------------------------------------------------------------------------------------------- segregation -------------------------------------------------------------------------------------------
Test Num             Population Accu      Gen1 Accu            Gen2 Accu            Gen3 Accu            Gen4 Accu            Gen5 Accu            Test Name                           
2                    0.838                nan                  nan                  0.972                0.970                0.962                ped_only                            
3                    0.700                nan                  -0.102               0.974                0.970                0.962                combined

We can see that the AlphaImpute2 results are much better in Gen3 and Gen4. But for the Gen2, ped_only cannot give any sensible output (it would be all 0.25s), combined is giving something negatively correlated to the true segregation probability. This is caused by the issue described above.

[XingerTang]

@gregorgorjanc

Genotype probabilities are not completely diffused. The founders' genotype data are mostly known. The phase might be difficult to resolve completely since we lack the data from the earlier generation, but we will at least be able to know which parent the haplotype inherited for some haplotypes for Gen2 individuals. But this part of the information is completely ignored for now.
In the case of AlphaImpute2, by considering genotype probability as AlphaPeel did, we can at least get a non-negative Gen2 accuracy.

[gregorgorjanc]

@XingerTang gotcha! Let's see how much information can we get out of these. How do you propose we estimate the seg probs between founders and their progeny?

[XingerTang]

@gregorgorjanc

The following example may be helpful in explaining how the genotype probabilities can help improve segregation outputs.

For the same example as in #51, we have the pedigree as follows:

1 0 0
2 0 0
3 1 2
4 1 3

and input genotypes as follows:

1 9 2 0 0 0
2 0 2 0 0 2
3 1 9 9 9 9
4 1 2 0 0 0

The output haplotypes are the following:

1 0 1 0 0 0
1 1 1 0 0 0
2 0 1 0 0 1
2 0 1 0 0 1
3 1 1 0 0 0
3 0 1 0 0 1
4 1 1 0 0 0
4 0 1 0 0 0

The output haplotypes are called from the genotype probabilities, while the segregation probabilities are calculated based on only the confidently proved haplotypes.

If we look at the haplotypes, for the progenies of founders 1 and 2, their progeny 3 has a very straightforward inheritance pattern, indicating that it inherits the grand maternal haplotype from individual 1. But its segregation outputs are all 0.25s:

3 0.2500 0.2500 0.2500 0.2500 0.2500
3 0.2500 0.2500 0.2500 0.2500 0.2500
3 0.2500 0.2500 0.2500 0.2500 0.2500
3 0.2500 0.2500 0.2500 0.2500 0.2500

While what is expected after we add the genotype probabilities information would be:

3 0.0000 0.0000 0.0000 0.0000 0.0000
3 0.0000 0.0000 0.0000 0.0000 0.0000
3 0.5000 0.5000 0.5000 0.5000 0.5000
3 0.5000 0.5000 0.5000 0.5000 0.5000

For the other progeny 4 of the founder 1, the phase of its first locus of the individual 4 is interpreted from the genotype probabilities, that the probability for 01 is 0.14, and the probability for 10 is 0.33. This phase suggests that the haplotypes are inherited from the grand maternal haplotype of individual 1 and the grand maternal haplotype of individual 3. The segregation probability output of that locus is

4 0.2824
4 0.2176
4 0.2824
4 0.2176

While what is expected after we add the genotype probabilities information would be:

4 0.0000
4 0.0000
4 0.0000
4 1.0000

[XingerTang]

I noticed that for the individual 4 example, if the phase is the other way round (i.e. 01 instead of 10), the resulting segregation would be pp, and it even makes more sense than the previous phasing and segregation results, that no mutation or recombination would be incurred at the last locus.

This is actually a special case that AlphaImpute2 is not confident in both of the phasing (the probability of one is 0.14, and the other is 0.33). I guess that just reminds us to be careful when we are interpreting these results and using them for improving segregation output.

[gregorgorjanc]

@XingerTang indeed - it's tricky to make calls under uncertainty! How should we handle this? Pick the most likely case? This is why I was interested if we can use max-product (Viterbi) algorithm to make such a call - see https://en.wikipedia.org/wiki/Viterbi_algorithm vs https://en.wikipedia.org/wiki/Belief_propagation