MIS with IBL

(L) [2017/07/06] [ost by jbikker] [MIS with IBL] Wayback!

Hi all,

I have a question about something that I can't seem to wrap my head around.

Here's the situation:

In a unidirectional path tracer with next event estimation and MIS lights can be reached by a random bounce and by explicitly sampling the light source. For MIS I thus take both pdfs into consideration, both when I stumble upon the light and when I sample it explicitly. So far so good.

Now, I add in IBL. In my first implementation, I randomly select either the skydome or one of the lights for next event estimation (50/50). When I stumble upon the light, I use the pdf for sampling a light scaled by 0.5 to account for the fact that I could have sampled it with 50% probability. Same for the sky. This produces a correct image, so I believe this method is correct and unbiased. Any other probability for light/sky works, including 0 and 100%.

Obviously, some parts of the scene benefit more from lights than the skydome. So, I want to change the probability of sampling the light or the sky depending on potential contribution. So I evaluate a random point on the skydome (importance sampled) and a random point on a random light. Based on these 'potential contributions' I determine the probability of sampling a light or the sky. This probability is recorded, so that when I stumble upon a light or the sky I can use it like I used the 0.5 in the previous scenario.

Sadly, this introduces bias, although subtle, and I can't figure out what I am doing wrong.

One thing that I realized is that when the random selected light happens to be behind the point that is being evaluated the potential contribution is 0, and therefore the sky will be selected with a probability of 1. However, another light may in fact be visible, so when I stumble upon it the MIS probability for sampling it explicitly is wrongly set to 0. So I clamped the light/sky probability to the range of 0.1..0.9 to ensure that both at least have a chance of being selected, but this feels wrong, bias remains (although the situation is better) and it looks like I am not understanding something here.

Any ideas?

(L) [2017/07/08] [ost by patlefort] [MIS with IBL] Wayback!

When your ray stumble upon a light source and you want to do MIS, you need to know the probability of it being selected. It's impossible to know that probability without having sampled random points on all other light sources in your sampling method. I'm not sure if doing that make sense and will converge to a correct result.
>> However, another light may in fact be visible, so when I stumble upon it the MIS probability for sampling it explicitly is wrongly set to 0.
I don't understand that part, if you have say 3 lights and one is behind the point, you'd be left selecting between 2 lights. The probability being I assume derived from some weighted sum.

(L) [2017/07/09] [ost by andersll] [MIS with IBL] Wayback!

>> One thing that I realized is that when the random selected light happens to be behind the point that is being evaluated the potential contribution is 0, and therefore the sky will be selected with a probability of 1. However, another light may in fact be visible, so when I stumble upon it the MIS probability for sampling it explicitly is wrongly set to 0. So I clamped the light/sky probability to the range of 0.1..0.9 to ensure that both at least have a chance of being selected, but this feels wrong, bias remains (although the situation is better) and it looks like I am not understanding something here.
If you have 3 lights, then you need to consider all 3 when weighting them for selection. How exactly are you generating your weights? The simplest way is just using the (luminous) power of each light, then you can just normalize them to get probabilities, but obviously you have to consider all the lights.

More complex schemes would approximate the irradiance, e.g. by calculating the solid angle subtended by the lights' shape (this would naturally handle your case of a light being behind the current vertex's hemisphere).

Even more complex schemes take into account the BSDF of the surface.

The tradeoff is of course, the more accurate your approximation when selecting a light, the lower your variance, but the more expensive each event is to find. You're trading poor performance with small numbers of lights for high performance with large numbers of lights.

(L) [2017/07/10] [ost by javor] [MIS with IBL] Wayback!

>> jbikker wrote:
One thing that I realized is that when the random selected light happens to be behind the point that is being evaluated the potential contribution is 0, and therefore the sky will be selected with a probability of 1.
This sounds like you are mixing up the sampling probability with the contribution of the light source. If you have selected the light source without taking the visibility (or being infront/to the back of the point being shaded) into account, then the probability of this sampling strategy should be non-zero (and the sky probability < 1).

Otherwise, if your sampling strategy only samples lights that are visible (or in the upper hemisphere), this should be reflected in the algorithm, e.g. by discarding the sampled light source until a valid one is found or all are discarded. In this scenario the probability of sampling the sky would be 1, only if none of the remaining light sources are "visible".

(L) [2017/07/10] [ost by jbikker] [MIS with IBL] Wayback!

Thanks for your replies.
>> patlefort wrote:When your ray stumble upon a light source and you want to do MIS, you need to know the probability of it being selected. It's impossible to know if that probability without having sampled random points on all other light sources in your sampling method. I'm not sure if doing that make sense and will converge to a correct result.
Currently all lights have an equal probability of being selected. So when I stumble upon a light I have two options:

1. The last vertex was specular, so there is no other strategy to sample the light, so the contribution is simply throughput * light color.
2. The last vertex was not specular, so we now have a second strategy, so MIS is used. For that I calculate the light pdf = dist^2 / solid angle, and the brdf pdf, which I stored when I did the diffuse bounce. The combined pdf is then simply the sum of these.

This part is working, and agrees with a reference renderer (not using next event estimation).

What also works is IBL by itself: I can sample it with next event estimation and importance sampling, and MIS works too (similar to what I just described). But when I want to combine IBL and lights things get tricky:

1. If I either sample a light *or* the skydome everything is clear; the probability of sampling the sky is now e.g. 50% of what it was, which causes no problems in MIS. Same for the lights. Any other probability also works just fine.
2. If however I chose a probability based on some heuristic (in my case: 'potential contribution') things go bad, even when I can reproduce that exact probability when I apply MIS, and even when it's never zero for a light or the skydome. That's the part that is puzzling me...
>> javor wrote:This sounds like you are mixing up the sampling probability with the contribution of the light source. If you have selected the light source without taking the visibility (or being infront/to the back of the point being shaded) into account, then the probability of this sampling strategy should be non-zero (and the sky probability < 1).
But isn't any (non-zero) probability fine, in principle? I am not skipping any lights, nor any part of the skydome.

Conformance to reference is checked by the way by averaging the energy for each pixel and reporting this value. This quickly converges (even if there's tons of noise) and gives a very good indication of the accuracy of an approach.

(L) [2017/07/11] [ost by stefan] [MIS with IBL] Wayback!

>> jbikker wrote:Obviously, some parts of the scene benefit more from lights than the skydome. So, I want to change the probability of sampling the light or the sky depending on potential contribution. So I evaluate a random point on the skydome (importance sampled) and a random point on a random light. Based on these 'potential contributions' I determine the probability of sampling a light or the sky. This probability is recorded, so that when I stumble upon a light or the sky I can use it like I used the 0.5 in the previous scenario.
Just to be sure, the sample you're taking to calculate the probabilities is being discarded right after that and not being reused?
>> Sadly, this introduces bias, although subtle, and I can't figure out what I am doing wrong.
Out of curiosity, how do you notice that there's bias?

(L) [2017/07/11] [ost by jbikker] [MIS with IBL] Wayback!

>> stefan wrote:Just to be sure, the sample you're taking to calculate the probabilities is being discarded right after that and not being reused?
Correct.
>> stefan wrote:Out of curiosity, how do you notice that there's bias?
Well the setup is a simple box with a single-sided quad light well below the ceiling. One side of the box is open, letting in illumination from the skydome. The walls can thus 'see' the light up to a certain height. With the reference path tracer, the transition from 'below light level' to 'above light level' is somewhat fuzzy. The approach where lights or the sky are sampled with a custom probability causes this line to be unnaturally hard. Apart from that the overall energy level appears to be correct.
box.jpg

[IMG #1]:Not scraped: /web/20190530195910im_/http://ompf2.com/download/file.php?id=262&sid=45ae5ad815fce4863e853eb0fbe3068b

(L) [2017/07/11] [ost by ingenious] [MIS with IBL] Wayback!

The problem is that the probability you use for choosing a light source is itself a random variable. If you write out the estimator you're building with your sampling techniques, its expected value won't be what you want. To get an unbiased estimator, you'll need to divide not by that random probability (which you got by sampling one random point on each light) but by its integral for all possible points on the lights, which is some unknown constant. Obtaining this in practice is obviously not feasible.

You can easily test whether this indeed is the problem by using a light selection probability that is only a function of deterministically known things at the shading point (i.e. every time you land at this point you will compute the exact same light selection probability).

(L) [2017/07/14] [ost by jbikker] [MIS with IBL] Wayback!

>> ingenious wrote:You can easily test whether this indeed is the problem by using a light selection probability that is only a function of deterministically known things at the shading point (i.e. every time you land at this point you will compute the exact same light selection probability).
That is indeed correct: if I use a fixed probability for sampling the light or the sky, everything is OK.
You are correct that the light sampling / sky sampling probability itself became a random variable... So I am basically falling in the 'Resampled Importance Sampling' trap again.

- Jacco.

MIS with IBL back

Board: Home Board index Raytracing General Development