Commit Graph

432 Commits

Author SHA1 Message Date
James Monteath
ee54a8ace7 Update all README to clearify intention or usage
Add snap configuration file used by Buildbot snap store steps1
2021-05-31 18:58:42 +02:00
James Monteath
74c7e21f6c Add and update README.md files for CI script removal 2021-05-28 19:46:53 +02:00
James Monteath
5baf1dddd5 Buildbot related files have been moved to own repository 2021-05-28 10:16:39 +02:00
Ray Molenkamp
62bff15377 Fix various Blender 3.0 versioning issues
This changes the following items:

- package name is now `blender-3.0.0-git.09eb04c0a865-windows64`
  rather than `blender-3.00.0-git.09eb04c0a865-windows64`
- Fix version resource for blender.exe not building
- Data directories are now `3.0\...` rather than `3.00\....`
- User prefs are now in:
  `c:\Users\users\AppData\Roaming\Blender Foundation\Blender\3.0\`
  rather than:
  `c:\Users\users\AppData\Roaming\Blender Foundation\Blender\3.00\`
- Updating startup & preferences from previous release
  has a special exception for 3.0 to check for 3.93 and older.

See T87532

Ref D10986
2021-04-16 11:23:34 +10:00
Jörg Müller
d33339ebf4 Audaspace: add support for PulseAudio on Linux
This adds PulseAudio as audio backend on Linux.
PulseAudio is the main audio engine used on most,
if not all, Linux distributions today.

Ref T86590
2021-03-16 23:21:45 +01:00
Jeroen Bakker
dd2b616320 Buildbot: Fixed crash when building RC builds
Building RC builds would parse `BLENDER_VERSION_CYCLE_NUMBER` that
doesn't exist anymore. It was removed by {D7748}.

This change removes it from `buildbot_utils.py`.
2021-01-25 11:53:35 +01:00
Brecht Van Lommel
1dc9650d20 Cycles: upgrade to OptiX SDK version 7.1 on buildbot
This means that NVIDIA driver version 450 or newer is now required for Cycles
OptiX rendering. This upgrade is required to take advantage of new features and
improvements.

Ref T81431
2021-01-12 11:51:42 +01:00
Sergey Sharybin
1176591574 Codesign: Allo non-zero exit code for signtool on Windows 2020-12-01 17:16:48 +01:00
Sergey Sharybin
a2e00236d0 Revert "Codesign: Versioning code to support older branches"
This reverts commit 9d172f007e.

Got a second thought and remembered why it was not done in the first place.
The issue here is that the server needs to communicate codesign result back
and that must happen within the new protocol. So if the client talks old
protocol it is possible to receieve data from it, but is not possible to
communicate result back to it.
2020-11-13 11:35:04 +01:00
Sergey Sharybin
9d172f007e Codesign: Versioning code to support older branches
Turns out it is easier to have suboptimal versioning code on the server
side than to deal with branches where changes are to be merged into.
2020-11-13 09:49:02 +01:00
Sergey Sharybin
fb4113defb Codesign: Report codesign errors from server to worker
Pass codesign errors (if any) from codesign buildbot server to the
buildbot worker, so that the latter one can abort build process if
the error happens. This solves issues when non-properly-notarized
DMG package gets uploaded to the buildbot website.
2020-11-12 10:12:56 +01:00
Patrick Mours
3bb3b26c8f Cycles: Add CUDA 11 build support
With this patch the build system checks whether the "CUDA10_NVCC_EXECUTABLE" CMake
variable is set and if so will use that to build sm_30 kernels. Similarily for sm_8x kernels it
checks "CUDA11_NVCC_EXECUTABLE". All other kernels are built using the default CUDA
toolkit. This makes it possible to use either the CUDA 10 or CUDA 11 toolkit by default and
only selectively use the other for the kernels where its a hard requirement.

Reviewed By: brecht

Differential Revision: https://developer.blender.org/D9179
2020-10-13 15:15:44 +02:00
Campbell Barton
41d2d6da0c Cleanup: pep8 (indentation, spacing, long lines) 2020-10-02 11:59:16 +10:00
Campbell Barton
bab9de2a52 Cleanup: pep8, blank lines 2020-10-02 11:59:16 +10:00
Brecht Van Lommel
a4a814a180 Fix buildbot macOS packaging error after recent changes 2020-08-04 17:13:09 +02:00
Brecht Van Lommel
9538e38704 Fix buildbot packaging error after recent changes 2020-08-04 16:54:18 +02:00
Brecht Van Lommel
16c7b6a7fe Buildbot: make code signing of packages optional with --codesign parameter
This is in preparation of doing builds per commit that will not be code signed.

Ref D8438

Differential Revision: https://developer.blender.org/D8451
2020-08-04 15:44:41 +02:00
Sergey Sharybin
7860fc9539 Codesign: Wait for unsigned directory to appear
Solves problem with different order of codesign server startup and
mount of network shares: avoids exception happening when server is
started prior to the mounts are ready.
2020-06-30 11:21:00 +02:00
Sergey Sharybin
53799d3ed2 Codesign: CLeanup, redundant semicolon 2020-06-30 11:20:37 +02:00
Sergey Sharybin
2e0ac1e0c4 Buildbot: Cleanup, remove unused script and change naming
Follow upstream convention.
2020-06-17 17:39:17 +02:00
Sergey Sharybin
efb1bf80df Buildbot: Cleanup, typo in logging 2020-06-16 18:12:15 +02:00
Sergey Sharybin
2c2fd9f036 Buildbot: Switch WIndows builds to MSVC 2019 2020-06-16 16:17:47 +02:00
Sergey Sharybin
76d50c51a3 Buildbot: Support multiple workers talking to single codesign server
Is achieved by replacing hard-coded signed/unsigned file names with
"<uuid>" which acts as a "request ID". This way multiple workers can
put their requests into a single directory without collisions. The
code sign server will handle the requests sequentially in an unknown
order.
2020-06-16 14:08:14 +02:00
Sergey Sharybin
21d862494b Buildbot: Don't use builder name in the build directory
The directory layout on worker goes as following:

  <Worker>
    <Builder Name>
      blender.git/
      build/
      install/
      lib/

Adding an extra <Builder Name> after build is redundant.

Differential Revision: https://developer.blender.org/D8045
2020-06-16 12:24:01 +02:00
Sergey Sharybin
9f5cc128d9 Buildbot: Switch to devtoolset-9
Famous "should just work", but will never know until actually attempted
 to be used.

Ref T76783.
2020-06-12 16:06:54 +02:00
Brecht Van Lommel
a86b5df005 Blender: change bugfix release versioning from a/b/c to .1/.2/.3
The file subversion is no longer used in the Python API or user interface,
and is now internal to Blender.

User interface, Python API and file I/O metadata now use more consistent
formatting for version numbers. Official releases use "2.83.0", "2.83.1",
and releases under development use "2.90.0 Alpha", "2.90.0 Beta".

Some Python add-ons may need to lower the Blender version in bl_info to
(2, 83, 0) or (2, 90, 0) if they used a subversion number higher than 0.
https://wiki.blender.org/wiki/Reference/Release_Notes/2.83/Python_API#Compatibility

This change is in preparation of LTS releases, and also brings us more
in line with semantic versioning.

Fixes T76058.

Differential Revision: https://developer.blender.org/D7748
2020-05-29 17:48:26 +02:00
Brecht Van Lommel
5159ba6b33 Fix (harmless) PCRE not found warning when configuring CMake on Linux
Differential Revision: https://developer.blender.org/D7309
2020-04-02 17:09:01 +02:00
Brecht Van Lommel
2aa938b0ae Cleanup: simplify Linux buildbot config, where default values already match
Ref D7309
2020-04-02 17:09:01 +02:00
Sergey Sharybin
9f48852ba4 Buildbot: Enable version character for development builds
Allows to have 2.82a as a beta version on buildbot.
2020-03-10 10:23:28 +01:00
Sergey Sharybin
1f8f4c8cfe Codesign: Make file watcher robust for network errors 2020-02-21 11:00:11 +01:00
Sergey Sharybin
20b47d77e1 Codesign: Remove archive with files after they were copied 2020-02-21 09:31:55 +01:00
Sergey Sharybin
37cdfac6b9 Codesign: Harden check for archive being ready for sign
Seems like sometimes files are being only partially ready, which makes it so there
are unsigned files, failing to deliver fully signed bundle.

Now expected archive file size is stored into stamp file and is checked against
size of the archive file on another side.

There are some bare prints used for debugging, would need to switch it to a proper
logger (or to be removed).
2020-02-17 18:52:38 +01:00
Campbell Barton
1135c2cd17 Cleanup: CMake formatting 2020-02-15 10:40:41 +11:00
Sergey Sharybin
5527cd4ba0 Codesign: Attempt to make macOS codesign more reliable
Is still sometimes .ready file appears prior to an actual archive.
2020-02-12 18:57:57 +01:00
Sergey Sharybin
ae433393af Codesign: Fix script for Windows
Apparently, there is no os.sync() on Windows.
2020-02-04 10:24:59 +01:00
Sergey Sharybin
5dc1183580 Codesign: Possible fix for stamp appearing prior to archive
From looking into builder's logs it seems that stamp file is picked
up prior to actual archive: sometimes worker reports missing archive
file, from a code path which is only possible if there is a stamp file.

Could be something with IO scheduling where bigger file is sent to
Samba server after smaller file.

Hopefully with this change this will not happen anymore.
2020-02-04 09:47:59 +01:00
Sergey Sharybin
26ba7573e6 Codesign: Fix wrong logging message 2020-02-03 17:53:03 +01:00
Sergey Sharybin
3125cfceec Codesign: Add codesign for macOS worker
Works similarly to Windows configuration where buildbot worker and
codesign machines are communicating with each other using network
drive.
2020-02-03 17:03:51 +01:00
Sergey Sharybin
bc3d7faab7 Buildbot: Specify path to macOS codesign 2020-02-03 14:07:21 +01:00
Sergey Sharybin
8285a12f4e Buildbot: Make archive naming shorter and cleaner
It changes name to be blender-<version>-linux64.

Since CentOS is used as a base host for builds there is no real need
in specifying libc version. Is unlikely anything older could be used
anyway.

Also make bitness to be the same as windows. It is something what
users will read easier.
2020-01-30 13:32:22 +01:00
Sergey Sharybin
6dcb4c9b4f Buildbot: Allow building on systems without scl
Makes it easier to verify changes on local machine without scl
before committing changes to repo.
2020-01-30 13:32:22 +01:00
Sergey Sharybin
306edb7477 Buildbot: Enable 16 threads for macOS worker as well 2020-01-30 10:24:57 +01:00
Sergey Sharybin
b683fcb628 Buildbot: Increase number of compile thread for Linux Buildbot 2020-01-29 13:41:04 +01:00
Sergey Sharybin
adb6be36e4 Buildbot: Correct timestamp argument for Windows codesign
The current authority we use RFC 3161 time stamp server,
so need to pass different command line argument.
2019-12-03 14:41:49 +01:00
Jens
d4835b88b2 Buildbot: Migrate package archive format for Linux from tar.bz2 to tar.xz
xz compresses 25% better than bz2, reducing download times and server load.
The numbers:

blender-2.80-linux-glibc217-x86_64.tar.bz2 (release): 134 886 174 bytes
with xz:    96 181 604 bytes (-28.7%)
with xz -9: 93 871 548 bytes (-30.4%)

blender-2.81-7c1fbe24ca33-linux-glibc217-x86_64.tar.bz2 (beta): 173 600 363 bytes
with xz:    133 100 664 bytes (-23.3%)
with xz -9: 129 534 124 bytes (-25.4%)

xz also decompresses more than twice as fast as bz2, however compression needs
four times as long (on my 7-year-old laptop 3-4 minutes instead of <1).

Also xz has become more common than bz2, e.g. Debian/Ubuntu deb packages have
been xz-compressed for years, so the dpkg package manager as well as systemd
and grub all depend on liblzma being present, whereas bz2 is becoming more and
more optional.

Current Linux archives also include the UID/GID of whatever user account
happens to be used for building by the blender.org infrastructure. If someone
then installs these archives as root e.g. to /usr/local/... and doesn't pay
full attention the files remain owned by a regular user, which is a serious
security issue. This patch fixes that by setting the UID/GID to 0.

Differential Revision: https://developer.blender.org/D6138
2019-11-28 11:41:25 +01:00
Sergey Sharybin
1fbca07634 Buildbot: Increaser codesign timelimit
It was possible that it would exceed when signing all the DLLs if the machine
is busy with some background tasks or when internet is slow.
2019-11-26 10:08:33 +01:00
Sergey Sharybin
c25a910e4e Buildbot: Explicitly disable code signer on Linux and macOS
The script requires Python 3.7 as a very minimum, and CentOS is
only 3.6.

On macOC there was an access to a None object, due to missing
implementation of code signer on this platform.
2019-11-14 10:40:28 +01:00
Sergey Sharybin
c73a99ef90 Initial implementation of code signing routines
This changes integrates code signing steps into a buildbot worker
process.

The configuration requires having a separate machine running with
a shared folder access between the signing machine and worker machine.

Actual signing is happening as a "POST-INSTALL" script run by CMake,
which allows to sign any binary which ends up in the final bundle.
Additionally, such way allows to avoid signing binaries in the build
folder (if we were signing as a built process, which iwas another
alternative).
Such complexity is needed on platforms which are using CPack to
generate final bundle: CPack runs INSTALL target into its own location,
so it is useless to run signing on a folder which is considered INSTALL
by the buildbot worker.

There is a signing script which can be used as a standalone tool,
making it possible to hook up signing for macOS's bundler.

There is a dummy Linux signer implementation, which can be activated
by returning True from mock_codesign in linux_code_signer.py.
Main purpose of this signer is to give an ability to develop the
scripts on Linux environment, without going to Windows VM.

The code is based on D6036 from Nathan Letwory.

Differential Revision: https://developer.blender.org/D6216
2019-11-13 09:24:41 +01:00
Sergey Sharybin
e71963a37e Buildbot: Ensure proper ABI is used
This wasn't an issue in the real buildbot environment since the
precompiled libraries are compiled with same ABI as the compiler
used for Blender build. But it was causing issues when building
Blender using buildbot scripts (for troubleshooting purposes)
on a machine with different default compiler ABI.

Usually ABI detection is happening in platform_unix.cmake when
detecting whether there are any precompiled libraries folder
available. This detection is not happening when library folder
is provided explicitly, expecting ABI to be setup explicitly
as well.
2019-11-06 13:50:43 +01:00
Brecht Van Lommel
b57a89f062 Buildbot: don't add branch prefix when building release branches 2019-10-11 13:46:24 +02:00