Direct3d 11 uses fixed point in rasterization.
The Rasterizer(15) must support 16.8 (integer.fraction) fixed point precision for x and y.
(Direct3D 11.3 Functional Specifications).
By using 16.8 bits as fixed point precision, x, y range of viewport is [-32768(-2^15), 32767(=2^15-1)] in direct3d 11.
Then, I have question of above specifications.
Rasterization uses edge function, that is as follow, to test pixels are inside the triangle and calculate degree of interpolation(= barycentric coordinates).
Edge function of line contains vertex 0, 1:
(p.x - v0.x) * (v1.y - v0.y) - (p.y - v0.y) * (v1.x - v0.x)
(p is pixel position, vn is vertex n position, assume cw)
Edge function has multiplication of component of coordinate. If range of components of coordinates are [-32768, 32767], it required 32.16 fixed point to avoid overflow.
** How to avoid overflow of fixed point by using 16.8 fixed point in rasterization with viewport range is [-32768, 32767]?**
I tried several methods to solve this problem. But can't solve it.
Several methods are as follow:
- Advance with adding difference of edge function
I calculate each difference of edge function as difference of x, y. and increase with edge function value of initial pixel to process rasterization. But edge function value of initial pixel is 32.16 fixed point. It hasn't difference with using 32.16 fixed point.
This try is thinking of mentioned method in below link. Because of lack of my English skill, I fail to understand it.
Could somebody explain it more detail?
https://fgiesen.wordpress.com/2013/02/08/triangle-rasterization-in-practice/ Last paragraph of Sub-pixel precision
- Advance with adding difference of edge function / (2 * triangle size)
Range of barycentric coordinates in triangle is [0, 1]. I use it, that is value of division of edge function by (2 * triangle size) is barycentric coordinates, to test position of triangle and interpolate without overflow in triangle.
But overflow is occurred in outside pixel of triangle. To rasterize, I calculate min, max coordinates of triangle and rasterize pixel in an axis aligned bounding box consists of them. If the size of the triangle is enough small, overflow can be occurred in outside pixel of triangle. So it also doesn't solution.
If I uses scanline rasterization with Bresenham line algorithm, can choice inner pixel of triangle. and overflow isn't occurred. But I don't try this because I think Direct3d 11 uses brute force method that is testing all pixels in an aabb covers triangle for efficient using of a GPU.