-
Notifications
You must be signed in to change notification settings - Fork 0
Open
Labels
Milestone
Description
Currently we round the results of matrix multiplications, but the math is still done in floating point on the GPU. Per the spec the GTE did fixed point math. Would this be too much accuracy (on top of probably being slower)?