The longer-term goal is to separate host-only scene update
from device update: make it possible to make kernel features
depend on actual scene state and flags.
This change makes it so shaders are compiled before kernel
load, making checks like "has_volume" available at the
kernel features calculation state.
No functional changes are expected at this point.
Differential Revision: https://developer.blender.org/D12795
Making object which uses volume shader invisible will mark the shader
as not having a volume, forcing re-compilation of the shader to bring
it back to a consistent state.
The compilation is happening as part of scene update, which needs to
know kernel features. So there is a feedback loop.
Use more relaxed way of knowing whether there is a volume in the
shader for the kernel features, which doesn't require shader to be
compiled first.
Solves issues from the report, but potentially causes extra memory
allocated if the volume part of graph is fully optimized out. This
downside is solvable, but would need to split scene update into two
steps (the one which requires on kernel, and the one which does not).
It will be an interesting project to tackle, but for a bug fix is
better to use simpler solution.
This includes much improved GPU rendering performance, viewport interactivity,
new shadow catcher, revamped sampling settings, subsurface scattering anisotropy,
new GPU volume sampling, improved PMJ sampling pattern, and more.
Some features have also been removed or changed, breaking backwards compatibility.
Including the removal of the OpenCL backend, for which alternatives are under
development.
Release notes and code docs:
https://wiki.blender.org/wiki/Reference/Release_Notes/3.0/Cycleshttps://wiki.blender.org/wiki/Source/Render/Cycles
Credits:
* Sergey Sharybin
* Brecht Van Lommel
* Patrick Mours (OptiX backend)
* Christophe Hery (subsurface scattering anisotropy)
* William Leeson (PMJ sampling pattern)
* Alaska (various fixes and tweaks)
* Thomas Dinges (various fixes)
For the full commit history, see the cycles-x branch. This squashes together
all the changes since intermediate changes would often fail building or tests.
Ref T87839, T87837, T87836
Fixes T90734, T89353, T80267, T80267, T77185, T69800
Shaders are only compiled if they are used by some other Node (Geometry, Light, etc.).
This usage detection is done before updating the Scene, however it fails at detecting
Shaders used by Procedurals not known to Cycles (e.g. ones defined by third party
applications), as Procedurals are only updated after the shaders are compiled.
To remedy this, we now use the Node reference counting mechanism to detect whether a
Shader is used and therefore should be compiled.
This removes `ShaderManager::update_shaders_used` as it is not needed anymore, however,
since it would also update the Shader ids, this is now performed in
`ShaderManager::device_update`, and a new virtual `device_update_specific` method was
added to handle device updates for SVM and OSL.
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D10965
For indirect light rays, don't assume any hit is opaque, rather if it has
transparency or emission do the shading but don't do any further bounces.
Naturally this is slower when there are transparent surfaces, however
without this cutout opacity doesn't give sensible results.
Differential Revision: https://developer.blender.org/D10985
Static initialization order was not guaranteed to be correct for node base
types. Now wrap all initialization in accessor functions to ensure the order
is correct.
Did not cause any known bug on Linux/macOS/Windows, but showed up on this
platform.
The Blender/Cycles XYZ color space has a D65 white point instead of E, and
this was not correctly accounted for both in the OpenColor config reading code
and the bundled config.
This meant that since the OpenColorIO v2 upgrade, the Linear ACES color space
was not working correctly, and other OpenColorIO configs defining
aces_interchange were not interpreted correctly.
Ref T84819
Build System
============
This is an API breaking new version, and the updated code only builds with
OpenColorIO 2.0 and later. Adding backwards compatibility was too complicated.
* Tinyxml was replaced with Expat, adding a new dependency.
* Yaml-cpp is now built as a dependency on Unix, as was already done on Windows.
* Removed currently unused LCMS code.
* Pystring remains built as part of OCIO itself, since it has no good build system.
* Linux and macOS check for the OpenColorIO verison, and disable it if too old.
Ref D10270
Processors and Transforms
=========================
CPU processors now need to be created to do CPU processing. These are cached
internally, but the cache lookup is not fast enough to execute per pixel or
texture sample, so for performance these are now also exposed in the C API.
The C API for transforms will no longer be needed afer all changes, so remove
it to simplify the API and fallback implementation.
Ref D10271
Display Transforms
==================
Needs a bit more manual work constructing the transform. LegacyViewingPipeline
could also have been used, but isn't really any simpler and since it's legacy
we better not rely on it.
We moved more logic into the opencolorio module, to simplify the API. There is
no need to wrap a dozen functions just to be able to do this in C rather than C++.
It's also tightly coupled to the GPU shader logic, and so should be in the same
module.
Ref D10271
GPU Display Shader
==================
To avoid baking exposure and gamma into the GLSL shader and requiring slow
recompiles when tweaking, we manually apply them in the shader. This leads
to some logic duplicaton between the CPU and GPU display processor, but it
seems unavoidable.
Caching was also changed. Previously this was done both on the imbuf and
opencolorio module levels. Now it's all done in the opencolorio module by
simply matching color space names. We no longer use cacheIDs from OpenColorIO
since computing them is expensive, and they are unlikely to match now that
more is baked into the shader code.
Shaders can now use multiple 2D textures, 3D textures and uniforms, rather
than a single 3D texture. So allocating and binding those adds some code.
Color space conversions for blending with overlays is now hardcoded in the
shader. This was using harcoded numbers anyway, if this every becomes a
general OpenColorIO transform it can be changed, but for now there is no
point to add code complexity.
Ref D10273
CIE XYZ
=======
We need standard CIE XYZ values for rendering effects like blackbody emission.
The relation to the scene linear role is based on OpenColorIO configuration.
In OpenColorIO 2.0 configs roles can no longer have the same name as color
spaces, which means our XYZ role and colorspace in the configuration give an
error.
Instead use the new standard aces_interchange role, which relates scene linear
to a known scene referred color space. Compatibility with the old XYZ role is
preserved, if the configuration file has no conflicting names.
Also includes a non-functional change to the configuraton file to use an
XYZ-to-ACES matrix instead of REC709-to-ACES, makes debugging a little easier
since the matrix is the same one we have in the code now and that is also
found easily in the ACES specs.
Ref D10274
The implementation is currently optimized to load animation sequences once
and then quickly scrubbing through them. Later on an option should be added
to optimize for memory usage and only load the current frame into memory.
Currently mesh and curve objects are supported, including support for UV and
vertex color attributes. Missing still is support for arbitrary attributes and
motion blur, as well as better handling of changing topology. Shader assignments
are made using FaceSets found in the Alembic archive.
The animation (and constant) data of the objects inside the Alembic archive is
loaded at once at the beginning of the render and kept inside a cache. At each
frame change we simply update the right socket of the corresponding Cycles node
if the data is animated. This allows for fast playback in the viewport
(depending on the scene size and compute power).
Note this is not yet exposed in the Blender UI, it's a feature that is still under
development and not ready for general use.
Ref T79174, D3089
Procedurals are nodes in the scene that can generate an arbitrary number of
other nodes at render time. This will be used to implement an Alembic procedural
that can load an Alembic file into Cycles nodes. In the future we also expect to
have a USD procedural.
Direct loading of such files at render time is a standard feature in other
production renderers. Reasons to support this are memory usage and performance,
delayed loading of heavy scene data until rendering, Cycles standalone rendering
using standard file formats beyond our XML files, and shared functionality for
Cycles integration in multiple 3D apps.
Ref T79174, D3089
This optimizes device updates (during user edits or frame changes in
the viewport) by avoiding unnecessary computations. To achieve this,
we use a combination of the sockets' update flags as well as some new
flags passed to the various managers when tagging for an update to tell
exactly what the tagging is for (e.g. shader was modified, object was
removed, etc.).
Besides avoiding recomputations, we also avoid resending to the devices
unmodified data arrays, thus reducing bandwidth usage. For OptiX and
Embree, BVH packing was also multithreaded.
The performance improvements may vary depending on the used device (CPU
or GPU), and the content of the scene. Simple scenes (e.g. with no adaptive
subdivision or volumes) rendered using OptiX will benefit from this work
the most.
On average, for a variety of animated scenes, this gives a 3x speedup.
Reviewed By: #cycles, brecht
Maniphest Tasks: T79174
Differential Revision: https://developer.blender.org/D9555
The shaders were not tagged for a needed geometry update when the displacement method was modified, neither were the Geometry and Object managers.
Reviewed By: kevindietrich
Maniphest Tasks: T75539
Differential Revision: https://developer.blender.org/D8896
This encapsulates Node socket members behind a set of specific methods;
as such it is no longer possible to directly access Node class members
from exporters and parts of Cycles.
The methods are defined via the NODE_SOCKET_API macros in `graph/
node.h`, and are for getting or setting a specific socket's value, as
well as querying or modifying the state of its update flag.
The setters will check whether the value has changed and tag the socket
as modified appropriately. This will let us know how a Node has changed
and what to update, which is the first concrete step toward a more
granular scene update system.
Since the setters will tag the Node sockets as modified when passed
different data, this patch also removes the various modified methods
on Nodes in favor of Node::is_modified which checks the sockets'
update flags status.
Reviewed By: brecht
Maniphest Tasks: T79174
Differential Revision: https://developer.blender.org/D8544
This encapsulates Node socket members behind a set of specific methods;
as such it is no longer possible to directly access Node class members
from exporters and parts of Cycles.
The methods are defined via the NODE_SOCKET_API macros in `graph/
node.h`, and are for getting or setting a specific socket's value, as
well as querying or modifying the state of its update flag.
The setters will check whether the value has changed and tag the socket
as modified appropriately. This will let us know how a Node has changed
and what to update, which is the first concrete step toward a more
granular scene update system.
Since the setters will tag the Node sockets as modified when passed
different data, this patch also removes the various `modified` methods
on Nodes in favor of `Node::is_modified` which checks the sockets'
update flags status.
Reviewed By: brecht
Maniphest Tasks: T79174
Differential Revision: https://developer.blender.org/D8544
Problem: the Blender synchronization process creates and tags nodes for usage. It does
this by directly adding and removing nodes from the scene data. If some node is not tagged
as used at the end of a synchronization, it then deletes the node from the scene. This poses
a problem when it comes to supporting procedural nodes who can create other nodes not known
by the Blender synchonization system, which will remove them.
Nodes now have a NodeOwner, which is set after creation. Those owners for now are the Scene
for scene level nodes and ShaderGraph for shader nodes. Instead of creating and deleting
nodes using `new` and `delete` explicitely, we now use `create_node` and `delete_node` methods
found on the owners. `delete_node` will assert that the owner is the right one.
Whenever a scene level node is created or deleted, the appropriate node manager is tagged for
an update, freeing this responsability from BlenderSync or other software exporters.
Concerning BlenderSync, the `id_maps` do not explicitely manipulate scene data anymore, they
only keep track of which nodes are used, employing the scene to create and delete them. To
achieve this, the ParticleSystem is now a Node, although it does not have any sockets.
This is part of T79131.
Reviewed By: #cycles, brecht
Maniphest Tasks: T79131
Differential Revision: https://developer.blender.org/D8540
This reverts commit 33ce0cb5a1.
Fix T77273: crash enabling portal lights. The optimization for background
updates can be added back later for 2.90 and 2.83.1.
Remove old code that added extra updates for shaders that have a dependency on
objects. The dependency graph can now tell Cycles when a material is affected by
an object transform.
By default it will now set the step size to the voxel size for smoke and
volume objects, and 1/10th the bounding box for procedural volume shaders.
New settings are:
* Scene render/preview step rate: to globally adjust detail and performance
* Material step rate: multiplied with auto detected per-object step size
* World step size: distance to steo for world shader
Differential Revision: https://developer.blender.org/D1777
This is mostly straightforward, but required some refactoring to ensure that
the default volume material does not always turn on the volume feature for GPU
rendering.
Custom render passes are added in the Shader AOVs panel in the view layer
settings, with a name and data type. In shader nodes, an AOV Output node
is then used to output either a value or color to the pass.
Arbitrary names can be used for these passes, as long as they don't conflict
with built-in passes that are enabled. The AOV Output node can be used in both
material and world shader nodes.
Implemented by Lukas, with tweaks by Brecht.
Differential Revision: https://developer.blender.org/D4837
When doing simple scenes the displacement shading failed during final
rendering when the displacement method is set to `Displacement + Bump`.
When this option is enabled the shader uses the Vector math
node. This node is part of the node group level 1. When doing simple
shading only using nodes that are part of the node group level
0 the shading was rendered black.
This only happened in final rendering as there the OpenCL programs are
optimized to save registries. Viewport rendering rendered correctly
Reviewed By: brecht
Differential Revision: https://developer.blender.org/D5859
These are the internal changes to Cycles, for Blender integration there are no
functional changes in this commit.
Images are converted to scene linear color space on file load, and on reading
from the OpenImageIO texture cache. 8-bit images are compressed with the sRGB
transfer function to avoid precision loss while keeping memory usages low. This
also means that for common cases of 8-bit sRGB images no conversion happens at
all on image loading.
Initial patch by Lukas, completed by Brecht.
Differential Revision: https://developer.blender.org/D3491
This allows for extra output passes that encode automatic object and material masks
for the entire scene. It is an implementation of the Cryptomatte standard as
introduced by Psyop. A good future extension would be to add a manifest to the
export and to do plenty of testing to ensure that it is fully compatible with other
renderers and compositing programs that use Cryptomatte.
Internally, it adds the ability for Cycles to have several passes of the same type
that are distinguished by their name.
Differential Revision: https://developer.blender.org/D3538
