Color space: when to multiply by the matrix back

(L) [2014/06/04] [ost by Chaos] [Color space: when to multiply by the matrix] Wayback!

Hi,

This topic may not be as technical or full of equations as the other ones, so please excuse me [SMILEY :)]

There is something I don't quite understand when using colour space transforms. Let's assume my renderer work in a given linear colour space called A. The renderer takes different kind of colour inputs, such as texture for materials, and textures for lights (HDR for example), which are all expressed in the colour space A.

Now let's assume I want to 'render in' another linear colour space, B. As far as I know, there is a 3x3 (number of RGB components) matrix going from A to B, which we will call AtoB. It seems like I have to ways to 'render in the colour space B':
- doing my render as previously, then multiply the output of the render by the matrix AtoB.
- multiply all the colour inputs by the matrix AtoB, so that all my colour inputs are now expressed in the colour space B.

Of course, both methods are going to give me something different. If I only consider directlighting, at some point the colour of a given rendered pixel is going to be proportional to the product of some material texture and light texture. In the first case, I would then go and multiply this result by the matrix AtoB, once. However, in the second case, the same pixel would actually contain the matrix multiplication twice, wouldn't it?

(And we're not even talking about differences because of spectral->RGB things happening in the render, or non-linearity when computing indirect lighting, or metamerism).

So clearly, there are (many...) things I don't quite understand here. I can think ok:
- both methods are 'valid', and give different results. But which one should I choose?
- I shouldn't multiply the 'light textures' by the matrix because these RGB values represent something else, in which case I will get the same direct lighting with both methods

In any case, as soon as I will render indirect, it feels like pre vs. post color space application will always yield different renders, wouldn't it?

Would you mind helping me on this issue? I asked some imaging guys, but they couldn't get their head around the 3D rendering part [SMILEY :(]

Cheers,
Jon

(L) [2014/06/04] [ost by MohamedSakr] [Color space: when to multiply by the matrix] Wayback!

>> Chaos wrote:Now let's assume I want to 'render in' another linear colour space, B. As far as I know, there is a 3x3 (number of RGB components) matrix going from A to B, which we will call AtoB. It seems like I have to ways to 'render in the colour space B':
- doing my render as previously, then multiply the output of the render by the matrix AtoB.
- multiply all the colour inputs by the matrix AtoB, so that all my colour inputs are now expressed in the colour space B.
my guess, second option should be the correct one [SMILEY :)]

how to think about it? very simple, it is pure math!!, think as if you are converting any other simple stuff "like pressure (F * A)", for the pressure case, you will need to unify your units "Newtons and meters", or if you will convert it, you may need to change every single aspect "for higher orders", this comes directly from the equation

so converting from the start "will make sure" that you are deriving the correct "unified" space from the beginning without wasting "dimensions" conversion

(L) [2014/06/09] [ost by Chaos] [Color space: when to multiply by the matrix] Wayback!

Thanks Mohamed! This is also something I've seen happening somewhere else.

It still feels weird though, since usually when working with different unit systems, things cancel out or contains squared dimensions. There is probably still something I don't understand, because I'm disturbed by the fact color space conversion doesn't look like it's closed under composition.

Cheers,
Jonathan

(L) [2014/06/09] [ost by tarlack] [Color space: when to multiply by the matrix] Wayback!

That feels strange yes, but maybe the reason why It seems not closed is that we do operations that do not support well linear transformations, namely the component-wise multiplication of vectors we do when applying the BSDF to the incident energy.

(L) [2014/06/10] [ost by Serendipity] [Color space: when to multiply by the matrix] Wayback!

First of all: As long as you render RGB your results will be wrong (except of some special cases). In order to do anything real you need to use spectral rendering, otherwise the metamerism will take care of everything being just wrong.

That being said: converting everything to a common colorspace can be tricky. In theory it should not matter in which colorspace you do your calculations, so both options are valid in my opinion. However the devil lies in the details: if your colors, for example, are defined in a wide gamut color space and you convert it to linear sRGB-like color space you will easily get colors that are outside the color space. This means you will get reflection values above 1, causing you to reflect more light than is received. So you either have to clamp your reflections "colors" in some way or your results will not be pyhsical plausible anymore. But clamping them will change your color and therefore make your rendering wrong.

On the other hand: Converting from sRGB to wide-gamut, do your calculation and convert back to sRGB should give you the same result as directly rendering in sRGB.

So both ways can give you better or worse color reproduction, depending on what your respective color spaces are.

(L) [2014/06/10] [ost by MohamedSakr] [Color space: when to multiply by the matrix] Wayback!

>> Chaos wrote:It still feels weird though, since usually when working with different unit systems, things cancel out or contains squared dimensions.
here is a simple example, when you get the reflected ray color which case do you calculate:
Color2 C = Color A * Color B;

or

Color C = Color A * Color B;
where A is incident ray color and B is surface color

here is the main loss of dimensions [SMILEY :)] "which you noted as 2 matrices instead of 1, now consider the bounced ray C hitting another surface, it will go to the next order Color3 and so on.."
and this is generally true, for example consider a scene where you calculate only direct illumination, if that scene takes 1 minute for direct illumination, it will take 10 minutes to calculate first bounce of indirect illumination, and 50 minutes for second bounce, and so on..."

NOTE:all of the above is "improvised" by my understanding, I don't claim it is correct or true, I hope someone who got any degree "they are too many on this forums [SMILEY :)] " to clarify the stuff and tell me if what I said is true or not

(L) [2014/06/11] [ost by Chaos] [Color space: when to multiply by the matrix] Wayback!

>> Serendipity wrote:First of all: As long as you render RGB your results will be wrong (except of some special cases). In order to do anything real you need to use spectral rendering, otherwise the metamerism will take care of everything being just wrong.
I totally agree here of course, but I'm not sure VFX companies are ready to use a spectral pipeline yet [SMILEY :)] However, if we apply color space transforms only when converting from spectral to 'rgb', then it seems like it 'solves' the issue (and yield a unique solution too).
 >> Serendipity wrote:On the other hand: Converting from sRGB to wide-gamut, do your calculation and convert back to sRGB should give you the same result as directly rendering in sRGB.
That's where I think I'm missing something! If you convert from sRGB to wide-gamut, then for me it means having a matrix multiplication for your reflectances and for your lights (two matrices), where converting back to sRGB would mean multiplying once by the invert matrix... resulting in one net matrix multiplication, wouldn't it? So it looks like that even ignoring out-of-range values, rendering in sRGB all along, or converting to-and-back-from a wide-gammut space would yield different results [SMILEY :(]
 >> tarlack wrote:That feels strange yes, but maybe the reason why It seems not closed is that we do operations that do not support well linear transformations, namely the component-wise multiplication of vectors we do when applying the BSDF to the incident energy.
Isn't that a theoretical issue? I mean, the component-wise multiplication of 3-vectors is a shortcut for a proper spectrum convolution, which has the same issue, doesn't it?

It feels like 'reflecting' an energy(?) spectrum is indeed adding a 'dimension', but I wouldn't expect this to happen, because it doesn't make much sense: what is the dimension of what you are seeing (that being a mixture of direct/1-bounced/2-bounced/x-bounced lighting)?

Cheers,
Jonathan

(L) [2014/06/11] [ost by tarlack] [Color space: when to multiply by the matrix] Wayback!

Well, the problem you have in my opinion is that in your discussion, you consider that a BSDF is represented by a color (or spectrum) and is in essence a color (or spectrum). This is not the case: in the strictest sense, the BSDF is an operator from spectrum space (or a given color space if not rendering in spectral) to spectral space (or color space), and a color-space BSDF should be represented by a matrix. This matrix is often assumed to be diagonal, which leads to the standard representation similar to the energy type and the component-wise multiplication. But with the matrix representation, the problem becomes evident: applying a linear transform (such as color space conversion) to a BSDF represented by a color should bring a matrix: the product of the conversion matrix with the diagonal matrix containing the color coefficients, and the problems come from the fact that we only keep the diagonal terms of the transformed BSDF matrix.

(L) [2014/06/11] [ost by ypoissant] [Color space: when to multiply by the matrix] Wayback!

My impression is that it is better to render in a color space that is as close to the input color spaces as possible and then transform to the final output color space. I don't see any improvements that can be gained in term of render precision in transforming the input color space into some renderer color spaces. Of course, the input color spaces need to be linearized, but that should be about it.

(L) [2014/06/12] [ost by tarlack] [Color space: when to multiply by the matrix] Wayback!

>>
My impression is that it is better to render in a color space that is as close to the input color spaces as possible
For sure, it minimizes the non-diagonal terms of the matrix which are then ignored. In my intuition (which has its flaws, as the bugs I got with BSDFs have proven a large number of times  [SMILEY :mrgreen:] ), as long as there is no fluorescence, incomplete linear color spaces can be used (such as sRGB), but what about fluorescence ? Couldn't this cause out-of-gammut colors to appear ? Would a complete color space such as XYZ be better for this specific case, as the matrix representation of BSDFs is necessary anyway ?