I'm going to deflate the marketing spin a little bit since large companies tend to go down the route of 'there was an impossible task and we use outside the box thinking to do what no one else could'.
Tracing batch rays, sorting them for better coherency and sorting the shading for coherency are not new ideas, but they are long over due to be used heavily in offline, cpu, ray traced rendering situations.
Large amounts of geometry have been done in movies for a long time, so Disney didn't really do anything that couldn't have been done before. All the same tracing a path of light one at a time while interlacing the shading is terrible for performance so it is great that Disney was able to reap the benefits. It is something that should have been obvious to anyone who understood cache performance in CPUs, but now at least people have something non-academic to point to.
Given that this is HN, I'd encourage reading and commenting on the substance of the technical paper rather than the marketing webpage.
Those ideas have indeed been around for a long time; the paper nicely cites all of the related research work. (Including among many others, a SIGGRAPH paper I wrote on the topic 18 years ago.)
I think the paper is excellent. First, there's a big gap between academic papers on a topic and the experience of actually building a real system that works for real movies. It's unusual for people in industry to take the time to write up their experiences building these systems, so I salute their making this contribution to general knowledge about rendering systems.
It's easy to carp about "all could have been done before"; that seems like an argument that could be applied to try to dismiss just about anything.
I'll just say that I don't want to dismiss anything except for exaggerated marketing. I think it is reasonable to say that I should ignore it though.
> I think the paper is excellent. First, there's a big gap between academic papers on a topic and the experience of actually building a real system that works for real movies. It's unusual for people in industry to take the time to write up their experiences building these systems, so I salute their making this contribution to general knowledge about rendering systems.
I'm %100 with you. I love seeing the results of cache coherency used in a real scenario. I was only trying to say that the actual imagery could have been done in other renderers, albeit with more pain and I would guess more attention to level of detail.
So I'm not trying to downplay anything except for the idea that the actual movie itself couldn't have been done without cache coherent batch ray tracing, when in reality it is a big optimization that came from a bit of a leap of faith, which I think is significant enough to pay attention to.
I completely agree with you that the information in that paper is excellent and I think it's amazing that Disney is so willing to share so much information about their renderer, even though their sister company Pixar sells a renderer.
The question though is whether or not this paper should have been published at EGSR. Personally I think that an EGSR paper should either propose a novel idea or should provide a good survey of the field. I don't think this paper succeeds at either of those. Their method is 'simply' a combination of already published ideas and I don't think that they do a very good job at comparing those existing ideas.
Personally, I think this paper would have been better suited as a talk at SIGGRAPH, a publication at JCGT or a technical report (like Pixar). So: great paper, wrong venue.
I'm actually really glad they got the paper out there as a small pdf (and I'm actually glad they compared it to other commercial renderers). The reason is that it shows very real world performance results. Not only that but cache coherency optimization results are shocking and illuminating to many people because it seems so counter intuitive when there technically aren't more instructions being ran.
Every paper I read now I am looking for all the things that were left out, how the comparisons have been changed for each scene to make that particular algorithm look good etc.
It answered a big question lingering in my mind that I haven't been able to actually try out.
From an engineering standpoint I think that Hyperion is an amazing renderer and I'd love to dig into its source code. From an academic standpoint however I think that their paper on it wasn't very interesting. I don't mind reading about the details of a production renderer, as there is a whole lot of info on writing production renderers. But I don't think that it should have been published at EGSR. There were no real new ideas in that paper that were worth publishing. The paper could have had some merit if they provided a good comparison between their architecture and other architectures, but sadly they decided to compare their renderer with other off the shelf renderers. Very little is known about the architectures of those renderers and as they all have their own implementations of acceleration structures, texturing and other shared components, they aren't comparable at all. It would have been much nicer if they implemented their algorithm in a framework such as Mitsuba, so that it could be compared with other algorithms.
I do think you should give them more credit on the engineering side though. Using large amounts of geometry has been done before, but renderers such as RenderMan and Arnold require that all geometry fits in the memory at once. Tracing large batches of rays at the same time allows you to load in or subdivide geometry on demand, which makes the size of the harddrive and the size of the individual meshes the limit, instead of the memory being the limit. Their scheme also allows for creating good ray packets, which clearly is needed if we want to reap the benefits of Moore's law, as those extra transistors mostly translate to wider SIMD vectors instead of higher clockspeeds and simply increasing the amount of children of a BVH node won't scale well. Reordering the rays allows you to create coherent packets out of incoherent rays and it allows you to efficiently handle motion blur, as all rays in a packet need to be in the same time interval, which allows you to efficiently make use of those wider SIMD vectors.
I'm certain they could have produced Big Hero 6 without Hyperion, but Hyperion gave their artists a lot of freedom by removing some of the technical limitations.
> There were no real new ideas in that paper that were worth publishing.
The real world speed improvements from specific methods are the significant part. There are many ideas floating around, not all of them work out once you discover all the comparisons and data omitted from an academic paper.
>I do think you should give them more credit on the engineering side though. Using large amounts of geometry has been done before, but renderers such as RenderMan and Arnold require that all geometry fits in the memory at once.
This is out of core tracing and is great that they implemented it, but neither out of core tracing and sorting rays are feats of engineering.
It has actually been done many times in both academics and commercial renderers from at least a decade ago. SIMD is also used to various extents in any decent renderer to varying degrees of success. Sorting into packets isn't the only way to use SIMD, or even the only way to use to scale SIMD use to wider lanes. Interestingly Skylake should have fast gather/scatter operations which will change the effectiveness of various techniques.
Basically good design choices and some practical knowledge from trial and error has been heavily exaggerated by marketing. I would guess that the actual render programmers would say the same. This isn't a breakthrough, it is a refinement and that's good enough. I love what they did, but to carry the torch that the impossible was made possible is disingenuous.
I say that based on what I've seen and claims I've heard - optix, prman unloading and loading procedurals, a renderer making that claim a long time ago (maybe finalrender?) In any event I don't know if any work particularly well, for the same reasons that Hyperion succeeds of course.
I'm actually more surprised that you would think of sorting and batching of rays as a feat of engineering, I can't imagine it seems all that difficult to you after all your experience.
Tracing batch rays, sorting them for better coherency and sorting the shading for coherency are not new ideas, but they are long over due to be used heavily in offline, cpu, ray traced rendering situations.
Large amounts of geometry have been done in movies for a long time, so Disney didn't really do anything that couldn't have been done before. All the same tracing a path of light one at a time while interlacing the shading is terrible for performance so it is great that Disney was able to reap the benefits. It is something that should have been obvious to anyone who understood cache performance in CPUs, but now at least people have something non-academic to point to.