Commit Graph

165 Commits

Author SHA1 Message Date
Hikari-no-Tenshi
8644d20331 Fix changelog generation after repo tool update 2023-11-14 18:35:49 +02:00
Lup Gabriel
3651050e15
createjson: use 12.1 url 2022-08-07 18:01:54 +03:00
Lup Gabriel
b29491732a
roomservice: get me some crDroid
based on https://github.com/PixelExperience/vendor_aosp/blob/twelve/build/tools/roomservice.py
@gwolf2u: adapted to crDroid
2022-08-02 14:23:58 +03:00
iusmac
878d2fb71d crdroid: Fix and improve changelog script
* 3 times faster
* fixed sorting by commit date
* fixed random build hangs due to unhandled bugs in 'repo forall'

Signed-off-by: iusmac <iusico.maxim@libero.it>
2022-07-21 15:09:42 +02:00
Lup Gabriel
1b00e88454 crdroid: Generate OTA json at build time
needs vendor_crDroidOTA cloned to <source root>/vendor/crDroidOTA

Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
2022-06-23 01:46:53 +05:30
Pranav Vashi
30c4f0dc2d crdroid: Add changelog script
Squashed:
ed37902026
3484fbf1d9
caeb656fae
4b20c6b833
00b5467b5c
83641d09ea
8c1b56f6b8
712068531c

Change-Id: Iaa749a64020fbec8218c216c54e040563dbf438a
Signed-off-by: Pranav Vashi <neobuddy89@gmail.com>
2022-06-22 20:22:22 +02:00
Michael W
39881f863a repopick: Limit commit counting
* For every commit, repopick checks the last commits for the presence of
  the commits to be picked
* In every project a change should go in, it calls "git rev-list --count"
  to find the maximum amount of commits to be searched, but it only cares
  if there are less (or equal) commits at all than to be checked
* Therefore, we can limit the counting to one more than we want to check
* This is relevant for example for fw/b, where there is a huge amount of
  changes and therefore a lot of time used to count

* Example: fw/b
  git rev-list --count HEAD: 46.693s
  git rev-list --count --max-count=1000 : 0.019s
* Real-life example:
  repopick -t qs-lightmode
  Old: 2m33.375s
  New: 0m6.657s

Change-Id: If0500574fb282e332996b606dd9926841f8e0e88
2022-06-15 20:57:30 +02:00
Timi Rautamäki
882e97e4ad roomservice: add repo to syncable repo list if dir doesn't exist
breakfast may get stuck if the first breakfast for a target is
interrupted before cloning but after adding repositories to local
manifest. Re-doing breakfast skips syncing the repositories if
they're added to the manifest even if not cloned.

Change-Id: Ifefd08fa6da8785c1d5de0b27ac1a08a782f21d6
2021-07-29 21:14:02 +02:00
LuK1337
8137c7c3bf roomservice: Set exit status if we fail to fetch data from github
Change-Id: I6f69f1df1ba7d28e68c5961b405a54e209643db3
2021-05-10 10:32:45 +02:00
Michael Bestas
151e17118c build: Relocate extract utils to tools/extract-utils
Change-Id: I13489b54298ebc2410c38ca9e16aa7121a28f997
2021-01-03 03:33:59 +01:00
Sebastiano Barezzi
85359e5b65
fixup! extract_utils: implement patchelf
* Move $PATCHELF exporting from oat2dex to setup_vendor
* Since it requires $HOST to be set, let's make it global, so oat2dex can also use it

Change-Id: I4556a3c19cd01c9b3a68d358d19a361217d9c3c1
2020-12-23 16:51:06 +01:00
Alessandro Astone
46b6e3a454 extract_utils: generate_prop_list: Ignore vdex/odex files too
Change-Id: Ibe77e8b93466a71d4d0c48ca8f4ff2e00c6e40af
(cherry picked from commit ee669788bc86330215f2444b3beb1f3870329c9e)
2020-12-15 19:22:12 +01:00
Volodymyr Zhdanov
a039a46b0d extract_utils: implement patchelf
Change-Id: I406f70ef1ce9ec7a0998f77439b7d3fa3ec4e62a
2020-11-26 20:40:02 +01:00
LuK1337
a574c010d2 repopick: Use project revision if available
* Fixes repopicking from repos synced under multiple paths + branches.

Change-Id: Icb165f3d9df67f1b11304e8d7ed955464dcb57bc
2020-11-23 10:55:24 +01:00
LuK1337
abb6597d30 extract_utils: Set filename_from_src: true for CLASS=ETC targets
Change-Id: I881f6a67d84d8de0a4bef17bf3c9eac049679c38
2020-10-06 19:29:53 +02:00
dianlujitao
37e088a2c9 extract_utils: Disable ELF check for shared library
* This is indeed a nice feature, but it's absolutely a bad idea to
   hardcode dependencies of prebuilt modules in proprietary-files.txt.

Change-Id: I8c2d75ff62c0c7862f40e777bcbad4d9cebc074c
2020-09-20 14:56:47 +01:00
Alexander Koskovich
411066c311 extract_utils: Fix system_ext framework blueprint generation.
Change-Id: I20e44be6ea48b9bd5b05055d3cf465f63c908463
2020-09-17 10:00:17 +02:00
Luca Stefani
0409f23119 extract_utils: Add support for system_ext
Change-Id: I0760cdd9bec1a0a0551aa68a60c34f9c00932668
2020-09-09 20:17:37 +02:00
Aayush Gupta
d395aa21eb repopick: Address PEP8 warnings regarding indentations and variable naming
- Format variables into snake case
- Add spacing between # and text
- Space after ','
- Test for membership with "not in" instead
- Remove too many empty lines

Ref:
[0]: https://www.python.org/dev/peps/pep-0008/#function-and-variable-names
[1]: https://www.python.org/dev/peps/pep-0008/#block-comments
[2]: http://pep8online.com/

Signed-off-by: Aayush Gupta <aayushgupta219@gmail.com>
Change-Id: I16dedf465674056571782921983eb3ec4276a1f8
2020-08-08 21:23:41 +02:00
Aayush Gupta
5fd8e2b1ee repopick: Use empty metavar parameter to fix help's indentation
Signed-off-by: Aayush Gupta <aayushgupta219@gmail.com>
Change-Id: I05329c3e5d234846a65f230f69d6b36aa3cc6fa1
2020-08-08 21:23:27 +02:00
Aaron Kling
7d6601cf02 repopick: Use upstream instead of revision to determine branch
This is to allow repopick to determine a branch even on a
revision locked manifest. If upstream is not set, it falls
back to revision.

Per the repo manifest documentation:
Attribute upstream: Name of the Git ref in which a sha1 can be
found. Used when syncing a revision locked manifest in -c mode to
avoid having to sync the entire ref space. Project elements not
setting their own upstream will inherit this value.

Change-Id: I12876f7e3b440f9eab6d1b96eba9b18a13cff2e0
2020-07-20 19:23:05 +02:00
Aaron Kling
d97bfec63f repopick: Fallback to name when project path is empty
When a manifest project path and project name is identical, the
repo manifest parser returns None for the project path. Since
both name and path are required, fall back to using name for path
when path is None.

Change-Id: I2fb3cc0cc643808a3049171804742f249d737679
2020-06-28 18:38:47 +02:00
Andreas Schneider
b1d3f65ac2 extract-utils: Fix usage for ETC files which need sub_dir
Thanks to Tad.

Change-Id: I6a0e7978afce6444d0827f97d7686f23aa7500c9
2020-05-25 23:52:52 +02:00
Andreas Schneider
a7e400ceb6 extract_utils: Fix vendor Android.bp creation with subdirectory
error: vendor/samsung/beyond0lte/Android.bp:37:9: unrecognized property "sub_dir"

Change-Id: I23becda0345f41d2cb1462e74b66a787e9ad4c96
2020-05-23 16:00:38 +02:00
dianlujitao
ee95f242c8 extract_utils: Handle Brotli compressed block image
Change-Id: I1b3ef34ebba9b43fd684c6abaa1dbc77c8786da7
2020-04-25 16:21:32 +02:00
dianlujitao
fc48634419 extract_utils: Support odm/product/vendor partition for zip extract
Change-Id: Iec06835f1703e276f8882c5ee327ef241a2f8f23
2020-04-25 16:21:32 +02:00
dianlujitao
0b501d534a extract_utils: Generate deodexed apk/jar with consistent checksum
* zip stores timestamp for all included files. The timestamps of dex
   files are different across different runs, result in inconsistent
   checksum of output apk/jar.
 * Workaround the issue by using fixed timestamp for dex files.

Change-Id: I21f3a7e32cdfdb07c5f5c140df2e797efd4a8005
2020-04-06 13:10:36 +08:00
dianlujitao
db1caf4597 extract_utils: Fix doubled path separator
* $DST_FILE is already prefixed with "/", don't double it.

Change-Id: I2655313abd7720a54b20f541a95960ae9ec373ad
2020-04-06 12:43:17 +08:00
Han Wang
ae82c34dd5 extract_utils: Fix up 4a2b65fb60
Change-Id: I9e5ea799812ca932de95ec6cbdedb604cbde5736
2020-03-10 09:41:34 +02:00
XiNGRZ
4a2b65fb60 extract_utils: Fix bad substitution for zsh
Change-Id: I5f4fabc62b1ac3da159b36c3ae6080e145948530
2020-03-10 07:49:55 +01:00
Tom Powell
f8adf067fd roomservice: support new manifest formats.
.repo/manifest.xml is no longer a symlink becuase apparently windows
developers need to use repo and windows needs admin for symlinks.

https: //gerrit.googlesource.com/git-repo/+/a269b1cb9dc21dfd598bfea3766206b606ad4589
Change-Id: I88ea0295133959136d7214f13a76b66d89dc88d4
2020-02-25 20:48:12 -08:00
dianlujitao
33ee59662f extract_utils: Drop string after semicolon when parsing destination
This fixes parsing when arguments contain colons, a typical usecase
would be:
-vendor/app/TimeService/TimeService.apk;:timeservice_app_cert

Change-Id: I7500ae09632632ddc10734d9b1df267e28286b67
2020-01-10 15:19:47 +01:00
Simon Shields
d5e35c9dda repopick: fix --check-picked on Python 3
This is very subtly broken: we look for the string 'Change-Id:'
in an array of byte strings. Fix this by decoding the git output
to utf-8 strings.

Change-Id: I708ad0adacb61c89bfba0fd88eeb2e37648317af
2019-11-22 16:48:05 +01:00
Adrian DC
045f664a1a repopick: Support project paths detection on different branches
* When some projects are declared in the manifests with specific
    changes (revision="refs/changes/../....../."), the path
    detection does not work, while most cases have a unique paths

 * Allow projects with unique branches to select their paths
    upon repopick with a warning about the selection

Change-Id: Ic873d69f57c78f233db3d0de4ebd529f896799ea
2019-10-30 12:59:48 +01:00
Mohd Faraz
d1d72351b2 extract_utils: Added Suffix uses COMMON_SUFFIX
* COMMON_JAVA_PACKAGE_SUFFIX for jar

 * COMMON_ANDROID_PACKAGE_SUFFIX for apk

Change-Id: I812405dac12ef7183985c66a6e43b0ea5f85989c
Signed-off-by: Mohd Faraz <mohd.faraz.abc@gmail.com>
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
34b5cdc1e3 extract_files: Implement blueprint file generation
Switch to blueprint on:
 - shared objects
 - $partiton/etc/ files
 - JARs
 - executable binaries and scripts
 - APKs

Only /sbin binaries are still in Android.mk because blueprint
doesn't handle sbin installation yet

Change-Id: I1dfd7e8bb575367b2a7fa9e333c4c6fa3aa68180
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
42752d4d4e extract_files: Add initial blueprint creation
Start the copyrights for these at 2019 because the blueprints won't
have existed before then.

Change-Id: I5300109949785dd19555e35d17a1e80daef498b0
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
8aae50df3c extract_files: Match non-treble path prefixes
Some devices put stuff on /system, /system/vendor or even
/system/vendor/odm. Search for these paths too when generating
TARGET_COPY_OUT_$partition variables.

Change-Id: Ie2c087e57aaca02d5ea93f290d5fc50d1315a600
2019-10-27 11:24:28 +00:00
Rashed Abdel-Tawab
8de76b204b extract_files: Support system/ prefixes
With support for 4 independent partitions now, we seriously
need to start putting /system blobs in their own directory.
Add support for file lists with system/ prefixes while
maintaining support for old file lists without it.

Also, TARGET_COPY_OUT_SYSTEM is a thing now, and all devices,
regardless of treble or not, set TARGET_COPY_OUT_$partition
so let's get rid of the treble compat option and default it
to true.

Change-Id: I5b798d293768d7c1e16db3ba01e2de3e083088d7
2019-10-27 11:24:06 +00:00
PIPIPIG233666
17d7dcba43
extract_utils: Fix typo
* ./../../xiaomi/sm6150-common/../../../vendor/lineage/build/tools/extract_utils.sh: line 1: /#!/bin/bash: No such file or directory.

Signed-off-by: PIPIPIG233666 <2212848813@qq.com>
Change-Id: I178f745d4ecb818c38706ff100611df19221065d
2019-09-29 12:49:44 -04:00
Bruno Martins
8194b8e583 extract_utils: Source the prebuilts from the new repo
The prebuilt tools now live in prebuilts/tools-lineage.

Change-Id: I4022210c52e1002aae0c6e622a480e5c7e10c366
2019-09-29 03:06:49 +02:00
Bruno Martins
3dff7da473 build/tools: Move prebuilts to the new repository
Going to be imported into prebuilts/tools-lineage.

Change-Id: I81dd06249a8de8f4970be30d75b396c85d4c4fb7
2019-09-29 03:06:49 +02:00
Rashed Abdel-Tawab
08e3a27135
extract_files: Add ODM partition blob support
Change-Id: Id94bbef879b6f4bcc20645c259981f9d163649c9
2019-09-27 06:46:44 -07:00
LuK1337
c62a9fbc31 repopick: Print change subject as unicode
* This prevents from seeing stuff like
  "b'frameworks: Add unlinked ringtone and notification volumes'"
  when using python3 as default.

Change-Id: Ie1fa85681b648edcee65680d784da4dff1779616
2019-09-21 11:48:49 +02:00
razorloves
b5c2c96a56 extract_utils: Support extracting from product partition
Change-Id: I2d0d01253fdf93d63a26556208d45cd69e76329f
(cherry picked from commit ea15be6e92e3be0c103948d220de64fd2e1d781f)
2019-09-10 11:28:05 +02:00
Rashed Abdel-Tawab
19c36cd7a6 extract_utils: Support multidex
Co-authored-by: Erfan Abdi <erfangplus@gmail.com>
Change-Id: I4881658d303b6c5c7f0b141beb50c1e9100af611
2019-07-18 07:31:36 +02:00
Rashed Abdel-Tawab
1c29c373dd extract_utils: Add functions to extract vendor blobs from vendor.img
Change-Id: I6f37adc955856fa78e719eb30249800abe80020c
2019-06-26 04:37:44 +02:00
Luca Weiss
5ee35ea356 repopick: cmp() is not available in Python 3, define it manually
Change-Id: I16a89aa555e62f105d888e87cb43afd26606076b
2019-06-08 00:01:05 +02:00
Vladimir Oltean
4818c230b3 extract_utils: introduce support for executing blob fixups
* Traditionally, the task of hex-editing blobs has been approached in 2 ways:
  (1) Do it out-of-band, commit the modified blob, and record its edited
      sha1sum in proprietary-files.txt (aka pin it).
  (2) Do it in-band, by adding code to the device-level extract-files.sh
      (usually this performs patchelf or sed). This code runs after the
      extract_utils functions were invoked.
* Problems of approach (1):
    - It relies on verbal (basically commit message) documentation of
      the hex-editing that was done. Makes it more difficult to reproduce.
    - Each time blobs are updated, pinning needs to be temporarily removed,
      hex-editing done again manually and new hash put back.
* Problems of approach (2):
    - It is incompatible with the concept of pinning, which is useful
      for kanging blobs from another device. A pinned blob would either:
        - Match the hash, get hex-edited, then it won't match the hash
          next time around.
        - Not match the hash (because of, say, hex-editing), then the
          extraction script would use an unwanted blob version instead of the
          pinned one (either that, or say "!! file not found in source").

* In summary, this patch adds system-wide support for approach (2) in order
  to address the aforementioned shortcomings.

* At device level, users of extract_utils who wish to perform blob
  fixups can override a blob_fixup() Bash function in their
  extract-files.sh immediately after running "source ${HELPER}". The
  blob_fixup() function will be called by the common extract() function
  after extracting every individual blob, giving the user the
  opportunity to hook custom code after this operation takes place.

* In proprietary-files.txt, the line corresponding to this blob which
  needs fixups can look in one of 2 ways:

(a) vendor/lib64/vendor.qti.gnss@1.0_vendor.so

    Do this if you are taking the blob from the stock ROM. The fixup
    script will always run after the blob is extracted.

(b) vendor/lib64/vendor.qti.gnss@1.0_vendor.so|249c76153f8de014bf2dd2ab623ee3d87741fbc8|f7e9ee8e3804887a2f3939128e860767e6f27258

    Do this if you are kanging the blob from somebody else. The pinning
    logic now applies for both the pre- and the post-fixup hashes. The
    fixup script will only run if the blob doesn't match the hex-edited blob,
    although the fixup script should really be idempotent.

Change-Id: Ifdd73c885d995c645f6210597537693d1a2f903f
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:44:53 +02:00
Vladimir Oltean
5238ba8fe5 extract_utils: make --section match words anywhere in the line
Change-Id: Ia3fe2160f0f4bc7da1b5bd31a21080af51b41fab
Signed-off-by: Vladimir Oltean <olteanv@gmail.com>
2019-03-03 16:42:24 +02:00