196

A lot of the implications for ray tracing are on the dev-side of things. It's a bit hard to explain without going into technical details.

Essentially, getting light to look "right" is very very hard. To do it, devs employ a lot of different techniques. One of those older techniques is baking the light on static objects, essentially pre-rendering where light goes and how it bounces. This has been done for a long time, e.g. even in Half-Life, the lights are baked for static geometry. So in a way, we have been using ray-tracing in games for a long time. however, it isn't real time ray-tracing, as the information gets stored in light map textures, so there is no performance impact other than storing the texture in RAM/VRAM and drawing the texture together with others.

The inherit problem of that technique is that it only really works for static geometry. If you move your light or any objects in the scene, your lightmaps will no longer match. To solve this, there are mixed modes which use real-time lights, dynamic light maps, and other tricks. However, these are often subject to problems and/or the limitations of using real-time lights. Real-time light problems are: You can only do a limited number before getting a serious performance impact, especially if the lights produce shadows. Soft shadows, shadows in big areas, and very detailed shadows are extremely hard to do as well without some advanced tricks. Also, ambient occlusion and global illumination is not something you can just give lights (there is screen-space GI and AO, but they don't look good in all circumstances, and you have limited control. There are also some other techniques some engines did for real time GI.).

Also there is the problem of baked light affecting dynamic objects, such as characters. This has been solved by baking so called "light probes". These are invisible spheres that store the light data and the closest data then gets applied to the characters and other dynamic objects. This again has a some problem, as it's hard to apply multiple light probes to the same object, so lighting might be off. Also, light direction is not accurate, which causes normal maps to look very flat in this light, and local shadows do not work using light probes. The same is done for reflections using reflection probes which are static. These are 360° "screen shots" essentially storing the reflection at that point in space. This however costs DiskSpace/RAM/VRAM, and it will not hold any information for moving objects (that's why sometimes you can't see yourself in the mirror in games). Also, the reflections sometimes look "out of place" or distorted when the reflection probe is too far from the reflecting surface (again, these cost VRAM and RAM so you don't want to place them in front of every single reflective surface). It costs a lot of time to find the right balance. For the rest, usually screen space reflections are used, as any other real-time reflection is extremely costly as you essentially render the whole scene again for each local reflection. Screen space reflection is an advanced technique that works very well for stuff like reflective floors, but you will quickly see its downsides on very mirrored surfaces as it lacks information that is not on the screen. Some games like Hitman for example use the mix of those techniques extremely well.

Coming back to lighting, there are now better techniques used for example by unreal and some other engines (and now unity in experimental). The light gets stored in more predictable data structures, such as 3d textures. This way, you can store the direction of all light in each cell. The light then gets applied to the objects passing through those cells. This looks pretty good, and the runtime cost is fairly low, but the storage cost of such textures is a tradeoff of texture resolution and fidelity. These textures cost a lot of VRAM to store and without using advanced techniques and tricks, have their own limits (e.g. for scene size). It also costs a lot of time to create each time you change the scene, and it also doesn't eliminate all problems mentioned above, like reflections, moving lights, etc.

Specifically, there is the problem of character lighting itself. Using light probes on characters usually looks pretty bad, as it removes a lot of detail of advanced skin shaders. Even with the above mentioned techniques, character lighting is still extremely hard to do. There is also some other problems, like ambient shadow in already shadowed areas, and light balancing for character versus scene lighting.

For that reason, most AAA games use separate light rigs for characters. Essentially floating lights that ONLY affect the character and move with them. When the mixing with the scene lights is done right, the rig adapts to the current situation in terms of light direction, color, and intensity. If you look in most AAA games, you can often see situations where rim-light comes from a direction where there is no actual light source. However, this way, the devs and artists have full control over lighting the characters. Essentially like a real movie production would have, but without the limitation of the real world.

Now, ray-tracing as you know it right now is not quite there yet, but eventually, ray tracing is the solution to a lot of the problems mentioned above. Things like polygon density, light count, global illumination, ambient occlusion, light direction, reflections, and much more are simply "there" for you to use. Now this doesn't mean that it will automatically make everything look great, but with the overwhelming amount of different tricks that have to be used for current gen games to make the look good, it opens a whole new world of possibilities.

Also, something that will not directly influence the final game, it will eventually simplify things for devs so that more time can be invested into other things.

At this current usage of ray-tracing, it's more like a gimmick, because devs will still focus most resources on the current ways to use light. This is because most people don't have cards with sufficient ray-tracing capabilities. So for the moment, I agree that the performance hit is not worth it. However, eventually it might become the default way to draw games. While we are not quite there, in terms of performance, I think that things might become a lot more consistent and predictable eventually for raytracing.

Remember PhysX back when it was a separate
card Physics Processing Unit before they shoved it on the GPU before they even had multithreading? Yea it evolved. But the original implementation was not ideal.

Cyberpunk 2077's RT Overdrive mode looks absolutely insanely good.

Raytracing is good but the problem is that were are in a transitional period (and Nvidia keeps upselling it's products)

I haven’t personally experienced a game that made use of it, so it must not exist

This u?

Yup, I have a 3090 and even then I don't bother with RTX. It's a gimmick Jensen and Nvidia love to push as a must have feature. In reality you don't notice it if you're playing a game normally, it's a "stop and smell the roses" feature you only turn on to check out once and turn off immediately when you get frame dips.

The problem with raytracing is that it's real strengths are in places where traditional rendering doesn't work at all. As soon as raytraced games stop needing a rasterized option, raytracing will really become useful. Most of it's advantages are around dynamic scenes where you can't just bake the lighting, or reflections which without raytracing will break if you look at them slightly wrong.

Edit: Most of the minecraft raytracing implementations are lacking in my opinion, but minecraft is a game that is well suited for raytracing. Really just anything with a dynamic world.

Spider-Man 2 has launched with ray tracing always on and looks and plays phenomenally. Super immersive to swing around the city and have proper reflections off all the skyscrapers!

Ray Traced particles are pretty cool in Returnal and Control.