The main focus here was to improve the docs significantly. Furthermore,
I reimplemented `Set`, `Map` and `VectorSet`. They are now (usually)
faster, simpler and more customizable. I also rewrote `Stack` to make
it more efficient by avoiding unnecessary copies.
Thanks to everyone who helped with constructive feedback.
Approved by brecht and sybren.
Differential Revision: https://developer.blender.org/D7931
This adds a new `CPPType` that encapsulates information about how to handle
instances of a specific data type. This is necessary for the function evaluation
system, which will be used to evaluate most of the particle node trees.
Furthermore, this adds an `IndexMask` class which offers a surprisingly useful
abstraction over an array containing unsigned integers. It makes two assumptions
about the underlying integer array:
* The integers are in ascending order.
* There are no duplicates.
`IndexMask` will be used to "select" certain particles that will be
processed in a data-oriented way. Sometimes, operations don't have to
be applied to all particles, but only some, those that are in the indexed by
the `IndexMask`. The two limitations imposed by an `IndexMask` allow for
better performance.
Reviewers: brecht
Differential Revision: https://developer.blender.org/D7957
Add TBB::flow graph scheduling to BLI_task.
Using flow graphs, a graph of nodes (tasks) and links can be defined.
Work can flow though the graph. During this process the execution of the nodes will be
scheduled among the available threads.
We are planning to use this to improve the threading in the draw manager.
The implemented API is still limited it only supports sequential flows. Joins and buffers
are not supported. We could eventually support them as part of an CPP API. These features
from uses compile time templates and are hard to make a clean C-API for this.
Reviewed By: Sergey Sharybin, Brecht van Lommel
Differential Revision: https://developer.blender.org/D7578
The changes come from the `functions` branch, where I'm using
these structures a lot.
This also includes a new `BLI::Optional<T>` type, which is similar
to `std::Optional<T>` which can be used when Blender starts using
C++17.
This is a more correct fix to the issue Brecht was fixing in D6600.
While the fix in that patch worked fine for linking it broke ASAN
runtime under some circumstances.
For example, `make full debug developer` would compile, but trying
to start blender will cause assert failure in ASAN (related on check
that ASAN is not running already).
Top-level idea: leave it to CMake to keep track of dependency graph.
The root of the issue comes to the fact that target like "blender" is
configured to use a lot of static libraries coming from Blender sources
and to use external static libraries. There is nothing which ensures
order between blender's and external libraries. Only order of blender
libraries is guaranteed.
It was possible that due to a cycle or other circumstances some of
blender libraries would have been passed to linker after libraries
it uses, causing linker errors.
For example, this order will likely fail:
libbf_blenfont.a libfreetype6.a libbf_blenfont.a
This change makes it so blender libraries are explicitly provided
their dependencies to an external libraries, which allows CMake to
ensure they are always linked against them.
General rule here: if bf_foo depends on an external library it is
to be provided to LIBS for bf_foo.
For example, if bf_blenkernel depends on opensubdiv then LIBS in
blenkernel's CMakeLists.txt is to include OPENSUBDIB_LIBRARIES.
The change is made based on searching for used include folders
such as OPENSUBDIV_INCLUDE_DIRS and adding corresponding libraries
to LIBS ion that CMakeLists.txt. Transitive dependencies are not
simplified by this approach, but I am not aware of any downside of
this: CMake should be smart enough to simplify them on its side.
And even if not, this shouldn't affect linking time.
Benefit of not relying on transitive dependencies is that build
system is more robust towards future changes. For example, if
bf_intern_opensubiv is no longer depends on OPENSUBDIV_LIBRARIES
and all such code is moved to bf_blenkernel this will not break
linking.
The not-so-trivial part is change to blender_add_lib (and its
version in Cycles). The complexity is caused by libraries being
provided as a single list argument which doesn't allow to use
different release and debug libraries on Windows. The idea is:
- Have every library prefixed as "optimized" or "debug" if
separation is needed (non-prefixed libraries will be considered
"generic").
- Loop through libraries passed to function and do simple parsing
which will look for "optimized" and "debug" words and specify
following library to corresponding category.
This isn't something particularly great. Alternative would be to
use target_link_libraries() directly, which sounds like more code
but which is more explicit and allows to have more flexibility
and control comparing to wrapper approach.
Tested the following configurations on Linux, macOS and Windows:
- make full debug developer
- make full release developer
- make lite debug developer
- make lite release developer
NOTE: Linux libraries needs to be compiled with D6641 applied,
otherwise, depending on configuration, it's possible to run into
duplicated zlib symbols error.
Differential Revision: https://developer.blender.org/D6642
This commit adds some new hashing based data structures to blenlib.
All of them use open addressing with probing currently.
Furthermore, they support small object optimization, but it is not
customizable yet. I'll add support for this when necessary.
The following main data structures are included:
**Set**
A collection of values, where every value must exist at most once.
This is similar to a Python `set`.
**SetVector**
A combination of a Set and a Vector. It supports fast search for
elements and maintains insertion order when there are no deletes.
All elements are stored in a continuous array. So they can be
iterated over using a normal `ArrayRef`.
**Map**
A set of key-value-pairs, where every key must exist at most once.
This is similar to a Python `dict`.
**StringMap**
A special map for the case when the keys are strings. This case is
fairly common and allows for some optimizations. Most importantly,
many unnecessary allocations can be avoided by storing strings in
a single buffer. Furthermore, the interface of this class uses
`StringRef` to avoid unnecessary conversions.
This commit is a continuation of rB369d5e8ad2bb7.
These two data structures reference strings somewhere in memory.
They do not own the referenced string. The string is considered
const.
A string referenced by StringRefNull can be expected to be
null-terminated. That is not the case for StringRef.
This commit is a continuation of rB369d5e8ad2bb7c2.
Many generic C++ data structures have been developed in the
functions branch. This commit merges a first chunk of them into
master. The following new data structures are included:
Array: Owns a memory buffer with a fixed size. It is different
from std::array in that the size is not part of the type.
ArrayRef: References an array owned by someone else. All elements
in the referenced array are considered to be const. This should
be the preferred parameter type for functions that take arrays
as input.
MutableArrayRef: References an array owned by someone else. The
elements in the referenced array can be changed.
IndexRange: Specifies a continuous range of integers with a start
and end index.
IntrusiveListBaseWrapper: A utility class that allows iterating
over ListBase instances where the prev and next pointer are
stored in the objects directly.
Stack: A stack implemented on top of a vector.
Vector: An array that can grow dynamically.
Allocators: Three allocator types are included that can be used
by the container types to support different use cases.
The Stack and Vector support small object optimization. So when
the amount of elements in them is below a certain threshold, no
memory allocation is performed.
Additionally, most methods have unit tests.
I'm merging this without normal code review, after I checked the
code roughly with Sergey, and after we talked about it with Brecht.
See Design task T68277, and patch D5423.
This commit includes edits by @ideasman42 to patch in
branch temp-D5423-update, plus responses to his comments.
Nothing special to mention about regression test itself, it basically
mimics the one for `BLI_task_parallel_mempool()`...
Basic performances benchmarks do not tell us much, besides the fact that
for very light processing of listbase, even with 100k items,
single-thread remains an order of magnitude faster than threaded code.
Synchronization is just way too expensive in that case with current
code. This should be partially solvable with much bigger (and
configurable) chunk sizes though (current ones are just ridiculous
for such cases ;) )...
The new data structure uses open addressing instead of chaining to resolve collisions in the hash table.
This new structure was never slower than the old implementation in my tests. Code that first inserts all edges and then iterates through all edges (e.g. to remove duplicates) benefits the most, because the `EdgeHashIterator` becomes a simple for loop over a continuous array.
Reviewer: campbellbarton
Differential Revision: D4050
This reverts reverting commit rB55324b8a2e6799300, and do proper 'cleanup' (sigh)
in gtest as well.
Sorry for the noise, did not understood what had happened here
immediately. :/
Simple isn't a good prefix for library names since
lots of unrelated modules could be called 'simple'.
Include 'py' in module name since this is a subset of Python,
one of the main motivations for this is to be Python like/compatible.
Recently @sergey found that hard-coding evaluation of certain very
common driver expressions without calling the Python interpreter
produces a 30-40% performance improvement. Since hard-coding is
obviously not suitable for production, I implemented a proper
parser and interpreter for simple arithmetic expressions in C.
The evaluator supports +, -, *, /, (), ==, !=, <, <=, >, >=,
and, or, not, ternary if; driver variables, frame, pi, True, False,
and a subset of standard math functions that seem most useful.
Booleans are represented as numbers, since within the supported
operation set it seems to be impossible to distinguish True/False
from 1.0/0.0. Boolean operations properly implement lazy evaluation
with jumps, and comparisons support chaining like 'a < b < c...'.
Expressions are parsed into a very simple stack machine program
that can then be safely evaluated in multiple threads.
Reviewers: sergey, campbellbarton
Differential Revision: https://developer.blender.org/D3698
It merely uses the new thread-safe iterators system of mempool, quite
straight forward.
Note that to avoid possible confusion with two void pointers as
parameters of the callback, a dummy opaque struct pointer is used
instead for the second parameter (pointer generated by iteration over
mempool), callback functions must explicitely convert it to expected
real type.
Also added a basic gtest for this new feature.
- Each allocation can be a different size
(but should be smaller than the chunk size).
- Result can be looped over in order of allocation.
- Allocations are aligned to pointer size to avoid unaligned reads.
This test should ensure we correctly detect all invalid utf-8 sequences in a given string.
DISCLAIMER:
Do not run this with current code - you'll either laugh or cry, nearly *all* checks fail!
Based on utf-8 decoder stress-test (https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-test.txt)
by Markus Kuhn <http://www.cl.cam.ac.uk/~mgk25/> - 2015-08-28 - CC BY 4.0
mat3_polar_decompose gives the right polar decomposition of given matrix,
as a pair (U, P) of matrices.
interp_m3_m3m3 uses that polar decomposition to perform a correct matrix interpolation,
even with non-uniformly scaled ones (where blend_m3_m3m3 would fail).
interp_m4_m4m4 just adds translation interpolation to the _m3 variant.
This commit:
* Adds a 'compare_ff' function for absolute 'almost equal' comparison of floats.
* Makes 'compare_vxvx' functions use that new 'compare_ff' one.
* Adds a 'compare_ff_relative' function for secured ulp-based relative comparison of floats.
* Adds matching 'compare_vxvx_relative' functions.
* Adds some basic tests for compare_ff_relative.
See https://randomascii.wordpress.com/2012/02/25/comparing-floating-point-numbers-2012-edition/
Note that we could replace our python/mathutils' EXPP_FloatsAreEqual() by BLI's compare_ff_relative
(using a very small absolute max_diff), but these do not have exact same behavior...
Left a comment there for now, we can do it later if/when we are sure it won't break anything!
Performance tests now have their own CMake macro, which ensures they do not get
added to ctest list, so we do not have to bother about them anymore, and can always
build them (when GTests are enabled, of course).
We could likely add much more, but those already covers basic behavior and should be able
to catch most errors when editing this code.
Also added some performances tests as well (timing ghash insert/lookup under heavy loads,
for different kinds of keys).
Murmur2a is a very fast hashing function generation int32 hashes.
It also features a very good distribution of generated hashes.
However, it is not endianness-agnostic, meaning it will usually generate
different hashes for a same key on big- and little-endian architectures.
Consequently, **it shall not be used to generate persistent hashes**
(never store them in .blend file e.g.).
This implementation supports incremental hashing, and is a direct
adaptation of reference implementation (in c++):
https://smhasher.googlecode.com/svn-history/r130/trunk/MurmurHash2.cpp
That cpp code was also used to generate reference values in gtests file.
Reviewers: sergey, campbellbarton
Reviewed By: campbellbarton
Projects: #bf_blender
Differential Revision: https://developer.blender.org/D892
Collection of test cases from libGDX and our own tracker
Tests:
- combine triangle area matches polygon area.
- tris have same winding.
- tris don't have duplicates.
- correct number of internal & boundary edges.
- degenerate polys still give topologically correct output.
also checks all possible start-vert offsets, forwards and backwards.
optional OBJ output, for debugging.
