Name: Protocol Buffers Short Name: protobuf URL: https://github.com/google/protobuf License: BSD License File: LICENSE Version: 21.12 CPEPrefix: cpe:/a:google:protobuf:21.12 Revision: f0dc78d7e6e331b8c6bb2d5283e06aa26883ca7c Security Critical: yes Shipped: yes Steps used to create the current version (or roll a newer one): 0. Nuke third_party/protobuf from orbit (rm -rf `third_party/protobuf`). Note that this removes this README.chromium so you may want it open in code search or remember the steps up until 2. where you get it back. 1. Pull the release from https://github.com/google/protobuf/releases (Source code .tar.gz). Extract to `third_party/protobuf` (without version suffix), i.e. you may `$ mv protobuf-21.12 third_party/protobuf` or similar. 2. Add back chromium's non-upstream files (from this directory): $ git checkout main BUILD.gn DEPS DIR_METADATA OWNERS README.chromium \ gen_chromium_file_lists.py gen_extra_chromium_files.py proto_library.gni \ patches/ Note: Ideally we wouldn't do this but instead have a `third_party/protobuf/src/` subdirectory where upstream code goes directly into. That will require updating things outside this repository (at least `third_party/webrtc`) to agree on where the source code lives. I wouldn't be surprised if that also required Google-internal uses of protobuf from webrtc to also agree on `third_party/protobuf/src` so maybe this'll never happen? Here be dragons. Probably. Probably worth making a local `git commit` here before applying patches (next step). 3. Apply patches in patches/ (see the description below): $ for patch in patches/*; do patch -s -p1 < $patch; done When a patch fails, it's probably best to check out previous state (from 3) for any changes not inside the patches/ folder, fix the patch and then try to reapply everything. It's not a good idea to fix the error, forget to generate a patch and leave patches/ in a bad state. If a patch no longer applies it may not be needed, if so remove it and its description in this README.chromium file. Similarly try to eventually upstream any patches out of band from unblocking the roll so we can eventually remove it here as well. Probably worth committing locally here too. 4. Generate `proto_sources.gni` using the script `./gen_chromium_file_lists.py`. Add and commit `proto_sources.gni`, 5. Generate descriptor_pb2.py using the script "gen_extra_chromium_files.py" in the same directory as this file. $ ./gen_extra_chromium_files.py -C ../../out/Default Add and commit `python/google/protobuf/descriptor_pb2.py` and `python/google/protobuf/compiler/plugin_pb2.py` as well. 6. Update Version, CPEPrefix and Revision in this file. Notes (hints) for the next roll(er): * This is probably harder to roll than most things you've rolled in the past. Good luck! Also please reach out for help proactively. The current roll as of writing this required help from multiple very-senior people to even figure out what was broken in the first place. `protobuf` rolling is not in a good place. * Update README.chromium as you go so you don't forget (after step 2. above, so you don't nuke those changes). * Rolling one version of protobuf likely introduces a large amount of changes. We should possibly figure out if it's reasonable to stay close to tip of tree and roll frequently rather than use released versions. Sharp corners here likely include that the version is encoded in protobuf output and I (pbos@) don't know if that version tag changes immediately on format incompatibilities or get bumped every time something format incompatible is introduced. * After v3.20 protobuf-javascript/ split out from upstream protobuf/. You may need to be aware of both repositories. Also upstream protobuf/ seems to move faster / out of sync with protobuf-javascript/. Hopefully that doesn't cause format incompatibilities in the future (protobuf/ supports something we need that protobuf-javascript/ doesn't). * Any argument changes to `protoc_wrapper` in `third_party/protobuf/BUILD.gn` may require updating `build/config/siso/protoc_wrapper.star` as well. * We've seen instances where instead of rolling protobuf a local copy of a new built-in type has been checked in. When that happens you may see ODR violations as your built-in type is defined in multiple files. If that happens you may want to move the copy that should never have been checked in into its own namespace, complete the roll and then cleanup the local copy. * third_party/mediapipe checks in a weird modified copy of `third_party/protobuf/src/google/protobuf/any.pb.cc` called `any_lite.pb.cc`. This precompiled protobuf will get invalidated on any protobuf version bumps and you'll need to either painstakingly generated a new one (do a diff of the checked-in `any_lite.pb.{cc,h}` and current `any.pb.{cc,h}` to aid you in the process. WARNING: This item is a lot larger than it looks. We should really either fix mediapipe upstream to build with `MEDIAPIPE_PROTO_LITE` and `MEDIAPIPE_PROTO_THIRD_PARTY` both set, but it looks like that configuration has bit rotted. Note about vpython: Some Python scripts end up mixing protoc output from this copy of protobuf with the google.protobuf module from vpython's protobuf. If Python scripts break due to the vpython copy of protobuf, you may need to update the version in //.vpython3. See https://crbug.com/1320047. Description of the patches: - 0004-fix-shared-library-exports.patch This patch allows exporting protobuf symbols in Linux .so libraries, so that protobuf can be built as a component (see http://crrev.com/179806). - 0008-uninline_get_empty_string.patch - 0010-uninline-generated-code.patch These patches uninline some functions, resulting in a significant reduction (somewhere between 500 KB and 1 MB) of binary size. - 0021-Fix-protobuf-s-library-.gitinore-file.patch Un-ignores python/google/protobuf/descriptor_pb2.py and python/google/protobuf/compiler/plugin_pb2.py - 0022-Allow-deprecated-fields.patch Allows deprecated fields to be used without extra C++ compiler warnings. - 0026-remove-sprintf.patch Imports https://github.com/protocolbuffers/protobuf/commit/c0fc2e881bc36aafb0bf539bf41889611370f60c to remove use of sprintf. - 0027-no-noreturn.patch Removes an instance of [[noreturn]]. The attribute is correct, but the way protobuf's GOOGLE_LOG(FATAL) is defined, the compiler can't see this and it trips -Winvalid-noreturn. See https://github.com/protocolbuffers/protobuf/issues/9817 - 0029-make-initializers-optimizable.patch Makes the InitProtobufDefaults() static initializer optimizable by Clang when built with libc++. It patches out the OnShutdownDestroyString call, which we do not need, and removes the thread-safe initialization. Thread safety is only needed if a static initializer spawns a thread which then calls InitProtobufDefaults() without synchronizing with the start of main(). (Anything which happens after main() starts can rely on the initializer running.) - 0030-workaround-window-constinit.patch Disables PROTOBUF_CONSTINIT in generated code in Windows shared library builds. Protobuf's default instances take pointers to a dllimport variable, fixed_address_empty_string. This is not constinit on Windows. This is a bug in protobuf as the default instance was intended to be constant-initialized. But the components build in Chromium is a developer configuration, so we tolerate an initializer as long as the build works, until protobuf has a proper fix. See https://github.com/protocolbuffers/protobuf/issues/10159. - 0031-workaround-cfi-unrelated-cast.patch A workaround for Clang's Control Flow Integrity check for casting pointers to memory that his not yet initialized to be of that type for empty arrays, does not work, and still fails. This patch removes the workaround and instead disables cfi-unrelated-cast for the affected methods and simplifies them. See https://github.com/protocolbuffers/protobuf/issues/10186. See https://bugs.chromium.org/p/chromium/issues/detail?id=1294200#c26. - 0032-cxx20.patch Fixes necessary to build in --std=c++20 mode. Imports https://critique.corp.google.com/cl/451177197 (a portion of https://github.com/protocolbuffers/protobuf/commit/6dd8af4ecfa7987bddb309862932886b84f1e4ef ). - 0033-no-enum-conversion-warn.patch Avoid hitting the clang error -Wenum-constexpr-conversion by specifying width of the enum. Imports https://critique.corp.google.com/cl/466986872. - 0034-change-macro-to-avoid-pedantic-warning.patch Avoid hitting -Wextra-semi. Imports (rebased): https://github.com/protocolbuffers/protobuf/commit/def602dd07b7eae1cac6823705975317b5607fc3 - 0035-fix-shared-library-constants Fixes for component build when building with MediaPipe. AnyMetadata::PackFrom is called which require string constants to be exported. - 0036-fix-undefined-memcpy-call.patch Fixes an undefined call to memcpy which may be passed NULL, 0. NULL, 0 is forbidden in C, due to a mistake in the C language specification. This partially integrates https://github.com/protocolbuffers/protobuf/commit/b2e1d7d8a1b1959c782595cd815c22fb690ac8e2 from upstream and may be removed when protobuf is updated past that commit. - 0037-fix-wextra-semi.patch Removes an extra semicolon, fixed in upstream https://github.com/protocolbuffers/protobuf/commit/3e6f8ee152db71ae49cc95649feab50d2faacdfa - 0038-silence-implicit-precision-loss.patch Silences -Wshorten-64-to-32 for now. Fixed in upstream https://github.com/protocolbuffers/protobuf/commit/d85c9944c55fb38f4eae149979a0f680ea125ecb - 0039-generate-mediapipe-anylite.patch Workaround to allow generating protobuf code compatible with third_party/mediapipe. This replaces checked-in precompiled .pb.cc files. See https://crbug.com/332939935, which when/if resolved should make this patch obsolete. - 0040-fix-ndk-major.patch Fix Android compile error that affects webrtc. Fixed in upstream https://github.com/protocolbuffers/protobuf/commit/9471a882256eaf01027524e8b6c15cbb5e42bef6