Related
So long story short, I wanted to build the ftdi_sio.ko module for the linux kernel of android based MiBox 1S. I was able to build the module but once I run "insmod" via adb, the system crashes and reboots (I couldn't see any error message because of this). Please could anyone suggest how to find where the problem is or whether what I wanted to do is even possible?
The device is running system version 1.3.88 rooted by the Chinese 360 root app. Kernel version follows:
Code:
adb shell cat /proc/version
Linux version 3.10.33 ([email protected]) (gcc version 4.7.3 20130205 (prerelease) (crosstool-NG linaro-1.13.1-4.7-2013.02-01-20130221 - Linaro GCC 2013.02) ) #1 SMP PREEMPT Tue Jun 16 00:38:26 CST 2015
As Xiaomi doesn't have kernel source release for MiBox (afaik), I went to look for the Amlogic kernel source for AML8726-MX (M6).
The closet kernel source I could find was from Stane1983's github repo amlogic-m6_m8. I'm using the earliest commit for 3.10.33 kernel version.
For the building of the kernel module, I pretty much followed a github document named 'Android-Build-Kernel-Modules-HOWTO.txt' by marco-pratesi. I downloaded the toolchain from the archive of linaro releases 13.02 which shows the same version of the android kernel above. I copied the .config from arch/arm/configs/meson6_deconfig and use 'make menuconfig' to add OTG_support and USB FTDI Single Port Serial Driver. Make the module by "make M=drivers/usb/serial/". And then adb push the generated ftdi_sio.ko file to /mnt/sdcard.
For loading the module, I run
Code:
adb shell
su
insmod /mnt/sdcard/ftdi_sio.ko
Then the box freezes for 10 seconds and reboots...
So my questions are:
1. Is there a way to find some logs of insmod to know what's gone wrong? As the system freezes and reboots, I can't use the dmesg method normally suggested, can I?
2. As the kernel source is from Amlogic, could it be that Xiaomi created their own kernel that is very different from this one to cause this problem?
3. Even if Xiaomi used exactly the same source code, as i'm using the meason6_defconfig file from the repo, it is likely to be different from the one Xiaomi was using. Would it matter for just building a module?
4. Is it possible to recover the config file from the prebuild kernel? I counld't find /proc/config.gz. And modprobe has error "can't open 'modules.dep': No such file or directory. "
5. What else I can try?
Sorry for the long post. I'm very new to this type of work and this forum as well. So any comments would be appreciated.
I am thinking of starting a new thread to solve the issue of ArnoldTheBat r72 booting to blank screen on Acer Chromebook C720P
oops I just did -
Building Chromium/Chrome OS r71 or r72 Kernel RFC
It seems the only way to tackle drivers issues is to build the Kernel.
So, trying to build a Chrome OS r72 Kernel to use on ATB r72 to address these drivers issues, particularly the inability to boot Acer Chromebook C720P -
As I have never done this before, I need some pointers as documentation for this is very sketchy on Google groups. RFC alesimula...
What I have available is Ubuntu Bionic, & necessary storage to build (over 100 GB) -
I also have a chromefied Nocturne then eve 73/swtpm.tar SSD (on top of ABT r72) with Bionic crouton with same over 100 GB storage available.
First steps to build Kernel
Build chrome os kernel and kernel modules
In a Chrome OS box (like Nocturne r71 with kernel 4.14) - install crouton Bionic
$ sudo sh ~/Downloads/crouton -r bionic -t xfce,xiwi,touch,extension,keyboard,cli-extra,chromium
after installation enter chroot
$ sudo enter-chroot
$ sudo apt-get install git-core make kernel-package bc nano
$ git clone https://chromium.googlesource.com/chromiumos/third_party/kernel -b chromeos-4.14
TBC after download
adapted from https://github.com/dnschneid/crouton/wiki/Build-chrome-os-kernel-and-kernel-modules
Kernel download Chrome OS Kernel 4.14
After issuing the git clone command - I received some errors...
such as
error: RPC failed; curl 56 GnuTLS recv error (-54): Error in the pull function.
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed
...
& after several tries succeeded with the message:
remote: Sending approximately 2.11 GiB ...
remote: Counting objects: 24648, done
remote: Total 7215674 (delta 5927776), reused 7215674 (delta 5927776)
Receiving objects: 100% (7215674/7215674), 2.10 GiB | 1.59 MiB/s, done.
Resolving deltas: 100% (5927776/5927776), done.
Checking out files: 100% (62736/62736), done.
So now the next business after securing the source code is to compile the kernel adding the modules/drivers needed to resolve issues such as camera & graphics, etc.
Hopefully this info is in here too
https://github.com/dnschneid/crouton/wiki/Build-chrome-os-kernel-and-kernel-modules
I am using Acer Iconia W700 with chromefied nocturne on top of arnoldthebat r72...
crouton bionic with xfce4 is my chroot environment to compile the kernel...
I guess I need to dig in, get the proper commands, it should not be different from compiling other Kernels for Android, Ubuntu, Arch Linux...
What I know is that it takes time...
hopefully if I get drivers, it will be worth the effort
Extra refs to solve cloning issues
https://devopscube.com/gnutls-handshake-failed-aws-codecommit/
https://stackoverflow.com/questions/38378914/git-error-rpc-failed-curl-56-gnutls
Compressed Chrome OS Kernel 4.14 using tar/xz is bleeming 2.4 GB!
$ tar cJvf kernel.tar.xz kernel
References for setting up configuration to build the Chrome OS kernel -
https://www.chromium.org/chromium-os/how-tos-and-troubleshooting/kernel-configuration
https://www.chromium.org/chromium-o...n-snow#TOC-Building-and-installing-the-kernel
Building Chrome OS kernel 4.14
Putting it together - trial & improvement (PC of trial & error)
in bionic chroot
sudo enter-chroot
go to kernel source code (folder where cloned)
$ cd kernel
$ ls chromeos/scripts/
generate-its-script.sh kernelconfig prepareconfig README splitconfig update_smatch_whitelist
challenge - how to generate .config to build kernel - & what are the commands to build it - I know how to with Ubuntu & Arch Linux, even done it a few time for Android Jelly Bean...
scratch - here goes...
from bionic chroot - kernel folder source code kernel/
sh ./chromeos/scripts/prepareconfig chromeos-intel-pineview
sudo modprobe configs; zless /proc/config.gz
cat /proc/config.gz | gunzip > ~/Downloads/base.config
base.config contains all the current configuration of ATB v72 kernel. This file should replace the current file on path kernel/chromeos/config/base.config
Editing base.config - to get extra kernel modules
I use nano
cd ~/Downloads/
nano base.config
it starts like this:
#
# Automatically generated file; DO NOT EDIT.
# Linux/x86_64 4.14.83 Kernel Configuration
#
CONFIG_64BIT=y
CONFIG_X86_64=y
CONFIG_X86=y
CONFIG_INSTRUCTION_DECODER=y
CONFIG_OUTPUT_FORMAT="elf64-x86-64"
CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig"
...
First change is Apple backlight keyboard replacing where it says is not set by:
CONFIG_BACKLIGHT_APPLE=m
Improving Kernel 4.14.83 from ATB v72 Methodology
Methodology
Extract base.config from FydeOS 5.3.1 (Chromium OS v70) & merge in its settings into base.config extracted from ATB Chromium OS v72...
Busy doing this, as I am not good at coding, so cannot devise an automated script, doing it manually is a long bummer
My comment in Telegram, frustration included -
Modifying kernel basic.config of ATB v72 to incorporate all settings/modules of FydeOS 5.31 (v70) is so tedious, it's a nightmare - I wish there was a way to automate this with a script that merges in FydeOS basic.config entries that are not present in ATB v72 - I am crap at coding & scripting, so doing it manually, it takes forever, & I keep making mistakes :stuck_out_tongue_closed_eyes: - as I am not sure what stops ATB v72 Acer C720P from booting, I have to include all missing settings in basic.config before building the kernel.
Chrome OS Kernel 4.14.96 built with extra modules for ATB v72
It does generate .config after all as usual for Linux kernels - got some issues from enter-chroot - had to do it from Ubuntu bionic proper with (from kernel source folder):
$ sh ./chromeos/scripts/prepareconfig chromeos-intel-pineview
$ make oldconfig
$ make -j4
Just now...
Finished compiling Chrome OS 4.14.96 for ATB v72 (which has Kernel 4.14.83), got vmlinux & modules - now need to find out how to deploy them on a Chrome OS installation...
After some struggle, managed to boot the new kernel as 4.14.96-09859-ga5c3f2f0428a-dirty
installed extra modules
by doing in crosh shell (not chroot)
in kernel compiled source code folder:
$ sudo make modules_install
it creates a new folder /lib/modules/4.14.96-09859-ga5c3f2f0428a-dirty
to get the kernel I just overwrite vmlinuz.A in /dev/sdb12 by kernel/arch/x86/boot/bzImage
(backup vmlinuz.A as vmlinuz.C)
to get to this:
$ mkdir efi
$ sudo mount /dev/sdb12 efi
$ cd efi
$ cd syslinux
$ sudo cp vmlinuz.A vmlinuz.C
$ sudo cp bzImage vmlinuz.A
../..
It boots great on Acer Iconia W700 & MacBook Air mid-2011
the joke is despite modules still no WiFi & no trackpad for MacBook Air
other joke does not boot Acer Chromebook C720P
well at least I tinkered with building this darn Chrome OS kernel - not much documentation on it (apart from how to do this in arm architecture)
Linux localhost 4.14.96-09859-ga5c3f2f0428a-dirty #1 SMP PREEMPT Wed Feb 13 08:04:55 GMT 2019 x86_64 Intel(R) Core(TM) i3-2365M CPU @ 1.40GHz GenuineIntel GNU/Linux
As I do not know how to proceed from this due to lack of Google documentation - I will stop.
I think to get it working, I need to install Chromium OS from scratch, but documentation is unclear, & arnoldthebat has no instructions on how to do this.
So I am starting an effort to build a full Chromium OS from scratch - as
the best way to learn is to share know how.
New thread - https://forum.xda-developers.com/hardware-hacking/chromebooks/chromium-os-building-t3900245
Chromium OS building effort here -
https://forum.xda-developers.com/hardware-hacking/chromebooks/chromium-os-building-t3900245
Rebuilding Chromium OS kernel - a little extra...
Small improvement - removing dirty label from kernel name -
in scripts/setlocalversion # comment out:
if git diff-index --name-only HEAD | grep -qv "^scripts/package"; then
printf '%s' -dirty
fi
will rebuild it again to see if this helps for deployment
Rebuilt successfully - works OK on Acer Iconia W700, including camera & camera migration...
Notes -
in chroot
sudo enter-chroot
in kernel/
sudo make modules_install
sudo make install
outside chroot
in kernel/
sudo mkdir /boot
sudo make modules_install
sudo make install
in ~/Downloads
sudo mount /dev/sdb12 efi
sudo cp efi/syslinux/vmlinuz.A efi/syslinux/vmlinuz.A.83
sudo cp kernel/arch/x86/boot/bzImage efi/syslinux/vmlinuz.A
& for backup
sudo cp efi/syslinux/vmlinuz.A efi/syslinux/vmlinuz.A .96
I think this wraps it up - sadly still not booting Acer CB C720P
Linux localhost 4.14.96-09859-ga5c3f2f0428a #4 SMP PREEMPT Sun Feb 17 11:02:41 GMT 2019 x86_64 Intel(R) Core(TM) i3-2365M CPU @ 1.40GHz GenuineIntel GNU/Linux
Not exactly... always something
Installing Chrome OS Kernel 4.1.4.96 removes crostini, wonder why that is...
solution might be flags - as new kernel, some flags might have been lost
chrome://flags
will check
crostini already there - must be some settings lost with new kernel, or dependencies unmet
NB - all previous commands presuppose you have r/w privilege as root, i.e. issue:
$ sudo mount -o remount,rw /
$ sudo mount -o remount,exec /mnt/stateful_partition
Note - my gripe after all this is that drivers I needed do not seem to load, or I did not select the proper entries in base.config / .config
What's the point of building a kernel if modules cannot be loaded
Google Chrome OS is well protected, & Chromium OS does not seem to bypass kernel protection.
nabil2000 said:
Methodology
Extract base.config from FydeOS 5.3.1 (Chromium OS v70) & merge in its settings into base.config extracted from ATB Chromium OS v72...
Busy doing this, as I am not good at coding, so cannot devise an automated script, doing it manually is a long bummer
My comment in Telegram, frustration included -
Modifying kernel basic.config of ATB v72 to incorporate all settings/modules of FydeOS 5.31 (v70) is so tedious, it's a nightmare - I wish there was a way to automate this with a script that merges in FydeOS basic.config entries that are not present in ATB v72 - I am crap at coding & scripting, so doing it manually, it takes forever, & I keep making mistakes :stuck_out_tongue_closed_eyes: - as I am not sure what stops ATB v72 Acer C720P from booting, I have to include all missing settings in basic.config before building the kernel.
Click to expand...
Click to collapse
hi, how did you extracted fydeos base.config ? if possible, can you provide me with a dropbox link to fydeos 5.3.1 base.config. Thanks
improving kernel 4.14.96
Hello2Clans said:
hi, how did you extracted fydeos base.config ? if possible, can you provide me with a dropbox link to fydeos 5.3.1 base.config. Thanks
Click to expand...
Click to collapse
Here is the dropbox link
[dropbox.com/s/esr407ybr5tev1u/configs.zip?dl=0](https://www.dropbox.com/s/esr407ybr5tev1u/configs.zip?dl=0)
it has the FydeOS 5.3.1 base config I extracted & your ATB v72 which I merged Fydeos 5.3.1 entries into - it updates to 4.14.96 from your 4.14.83 (replace z.config by .config)
It does compile a kernel OK - I did this manually, still learning how to do it signed. As I said details in my XDA thread.
I added all LCD panels modules, maybe it will finally allow to boot Chromebook C720P to GUI.
Oddly, I lose crostini, maybe unsigned kernels do that.
Linux localhost 4.14.96-09859-ga5c3f2f0428a #4 SMP PREEMPT Sun Feb 17 11:02:41 GMT 2019 x86_64 Intel(R) Core(TM) i3-2365M CPU @ 1.40GHz GenuineIntel GNU/Linux
You are welcome to improve on it - it builds, but modules I thought added did not work on MacBook Air nor graphics of Acer CB C720P
Hello2Clans said:
hi, how did you extracted fydeos base.config ? if possible, can you provide me with a dropbox link to fydeos 5.3.1 base.config. Thanks
Click to expand...
Click to collapse
I show how to do it in a previous post - but you need to be in a running FydeOS box, same with ATB.
sudo modprobe configs; zless /proc/config.gz
cat /proc/config.gz | gunzip > ~/Downloads/base.config
.config for Acer CB C720P
Still no luck with Acer Chromebook C720P - tried several changes to .config with make menuconfig & manually - boots to a blank screen still
Fydeos 5.3.1 does not have this problem.
Also tried just the .config of FydeOS on its own, & strangely no luck - there must be something else -
will now try this
# Display Panels
CONFIG_DRM_PANEL_LVDS=y
scratch - learning about kernel blobs /dev/sdx2 & /dev/sdx4
Just building the kernel & forcing in vmlinuz.A in /dev/sdx12 syslinux using bzImage (& modules in /lib/modules)
is apparently not enough...
It seems I also need to generate a new blob for the kernel which is /dev/sdx2
From documentation -
https://chromium.googlesource.com/chromiumos/docs/+/master/kernel_faq.md
we have
Kernel Root
pair A /dev/sda2 /dev/sda3
pair B /dev/sda4 /dev/sda5
in my case (as installed to usb)
Kernel Root
pair A /dev/sdb2 /dev/sdb3
pair B /dev/sdb4 /dev/sdb5
Some pointers here:
https://groups.google.com/a/chromium.org/forum/#!topic/chromium-os-dev/zmaziTddu5E
Major learning curve for me here...
what I need to do is upgrade the Arnoldthebat kernel blob KERN-A (4.14.83) with a new one for the new kernel (4.14.96).
I would appreciate if someone knew how to do this - that might be the cause issue of losing crostini when loading the new 4.14.96 instead of the original ATB.v72 4.14.83 one.
Share knowledge if you know how to do this, thanks.
Cooking -
I just flashed FydeOS 5.3.1 ROOT-A /dev/sdx2 blob which is 16MB over ATB v72 ROOT-A /dev/sdx2 blob which is 64MB & it booted, might be the solution I was looking for...
FydeOS (v70) is more compatible than ATB (v72) for many things. If the blob is involved, then it could be good news, easier than producing a new blob from scratch (don't know how either).
If so, working on blobs is an addition to-do list for hacking Chrome OS.
Hello. I am really glad someone is trying to build and install a custom kernel for ChromiumOS, as it will help fix *many* compatibility issues as well as it will make possible to add support for some hardware and features (for instance, audio over Bluetooth doesn't work as of now because the support library needed for BlueZ isn't included).
My computer isn't working properly and I don't have much space to build the kernel by my own, but I can help with testing and research. If you could share the binaries online (maybe a Git with releases?), this would be great.
What I have found and read about kernel on ChromiumOS so far:
1. Official documentation, it explains about the partition pairs:
http://www.chromium.org/chromium-os/chromiumos-design-docs/disk-format
2. More info (I think you have it already) but I think it is outdated as "console=tty1" doesn't work anymore (ChromiumOS now uses Frecon):
https://chromium.googlesource.com/chromiumos/docs/+/master/kernel_faq.md
3. This is a nice repo, it explains a lot, specially about kernel modules (drivers):
https://github.com/dnschneid/crouton/wiki/Build-chrome-os-kernel-and-kernel-modules
From what I have used and tested, you can use GRIB to either boot from the vmlinuz.A/B image or use the partition with the kernel, using the partition is much complicated as you found out because it would need some blobs/keys.
lfom said:
Hello. I am really glad someone is trying to build and install a custom kernel for ChromiumOS, as it will help fix *many* compatibility issues as well as it will make possible to add support for some hardware and features (for instance, audio over Bluetooth doesn't work as of now because the support library needed for BlueZ isn't included).
My computer isn't working properly and I don't have much space to build the kernel by my own, but I can help with testing and research. If you could share the binaries online (maybe a Git with releases?), this would be great.
What I have found and read about kernel on ChromiumOS so far:
1. Official documentation, it explains about the partition pairs:
http://www.chromium.org/chromium-os/chromiumos-design-docs/disk-format
2. More info (I think you have it already) but I think it is outdated as "console=tty1" doesn't work anymore (ChromiumOS now uses Frecon):
https://chromium.googlesource.com/chromiumos/docs/+/master/kernel_faq.md
3. This is a nice repo, it explains a lot, specially about kernel modules (drivers):
https://github.com/dnschneid/crouton/wiki/Build-chrome-os-kernel-and-kernel-modules
From what I have used and tested, you can use GRIB to either boot from the vmlinuz.A/B image or use the partition with the kernel, using the partition is much complicated as you found out because it would need some blobs/keys.
Click to expand...
Click to collapse
Thanks for the resources, I am getting old, so rather hard to learn & deal with new stuff - tinkering OSes & software is a hobby for me, but I usually do test stuff, not create - I had nice experience with Hackintosh, before I owned my own Macbook Air, still use it from USB's & PC when needed.
So far, I find out blob does not improve matters, it just gets used, like above blob from ATB or FydeOS does same thing.
The way I started in this was just to get Acer Chromebook C720P to load Android, chromefy carried on better than where I started.
The most successful outcome for me is Acer Iconia W700 which works with eve 73 (leaked on Telegram)...
I have the resources to build the kernel - but it is not doing what I want, like booting ATB v72 in Acer CB C720P - it i not vital, C720P works well with FydeOS & cyan or eve 71, just a challenge - why is ATB v72 not booting properly - i's a graphics VESA or LVDS bios issue I believe (C720P board uses LVDS for graphics), but where does it get loaded, I believe it's before loading modules...
FydeOS 5.31 boots to a black screen on my Miix320 but I have to wait until it loads everything then I must hit Ctrl+Alt+F2 then go back to main screen and then it works properly thereafter. Is your Acer the one bellow? Maybe it's related?
http://www.chromebookspecs.com/acer-c720p-chromebook
By the other hand, I cannot enable clicking for the detachable touchpad (+keyboard), what is almost a dealbreaker... Neither it goes to tablet mode when I detach it from the keyboard (not a big deal).
By the other hand, it boots correctly on a ThinkPad 8 tablet, but if freezes after a few seconds unless I use acpi=off or noacpi as kernel parameters, what makes it usesless since it only detects USB hardware.
From the text I linked it seems that the extra data added to the kernel before flashing it to a disk partition is both its signature and kernel paramenters, so you may want to check if there is any special paramenter needed for the kernel to boot correctly on your C720P.
FydeOS base upgrading kernel from 4.14.67 to 4.14.96
Kernel 4.14.96 upgrade progress -
I allowed
CONFIG_TCG_VTPM_PROXY=m
in .config
& this allowed me to update a chromefied FydeOS with eve/caroline 71 to eve 73 dev by using chromefy2!
this in turn allowed me to use an edimax Wifi USB dongle in MacBook Air -
the onboard WiFi still not loading, no trackpad but still progress :relaxed:
so modules do get loaded unlike my concern, it's just a matter to identify the proper entry in .config to load them when compiling the kernel...
now what are the proper entries for internal WiFi & trackpad?
Extra - Icing on the cake -
FydeOS 5.3.1 with eve dev 73 - putting the correct files in place allows to load kernel 4.14.83 & its modules to get crostini -
of course loading FydeOS kernel which is 4.14.67 stops Android 9.
This amended chromefy.sh script might help with MacBooks for trackpad & WiFi.
https://github.com/youngyou/chromefy/blob/master/chromefy.sh
*** Important note ***
To install the kernel modules in the Chrome OS box, you need to install Chromebrew to have access to the necessary commands -
$ curl -Ls http://git.io/vddgY | bash
In kernel source code folder:
$ sudo make modules_install
then copy bzImage over vmlinuz.A (after backing up the original)
I am planning to release an archive with kernels & modules including the one I built, it will be
vmlinuz.A.67(4.14.67); vmlinuz.A.83 (4.14.83); vmlinuz.A.96 (4.14.96); & corresponding /lib/modules folder
FydeOS 5.3.1 - ATB v72 - my built kernel
Notes -
Kernel 4.14.96 upgrade progress - I allowed CONFIG_TCG_VTPM_PROXY=m so this allowed me to update a chromefied FydeOS to eve 73 dev by using chromefy2! this in turn allowed me to use an edimax Wifi USB dongle in MacBook Air - the onboard WiFi still not loading, no trackpad but still progress :relaxed:
Icing on the cake - FydeOS 5.3.1 with eve dev 73 - putting the correct files in place allows to load kernel 4.14.83 & its modules to get crostini - of course loading FydeOS kernel which is 4.14.67 stops Android 9.
Cool progress - I managed to boot to the GUI of Acer CB C720P with eve dev v73 - still need to iron out something, but could get into guest mode, & to linux prompt - this is great :stuck_out_tongue_winking_eye: this is using kernel 4.14.83 on top of FydeOS 5.3.1
Finally managed to log into my google account in Acer CB C720P- next hurdle, will Android 9 work? - crostini back in the menu -
hurdle - despite changing vmlinuz.A to 4.14.83 or 4.14.96 - it still only loads FydeOS 4.14.67
so
I learnt something new, which means I need to learn more -
when you boot to a real chromebook from usb or internal, it will look for the kernel in the blob KERN-A /dev/sdx2, & will not use the vmlinuz.A in /dev/sdx12 -
so with FydeOS stuck with kernel 4.14.67 -
if I boot from other laptop, blob is not used, vmlinuz.A instead -
so I need to produce new KERN-A /dev/sdx2 blob for new kernel,
FydeOS blob is 16MB, ATB v72 blob is 64MB,
so to flash I need to play with partitions, tricky, & did not work...
still didn't figure out how to make a new blob for a new kernel...
RFC - how to produce a blob for loading a new kernel?
RFC - how to produce a blob for loading a new kernel?
If there is a way to find out how to produce a blob of an updated kernel for a genuine vanilla chromebook such as Acer Chromebook C720P - I will post it here...
Google open source documentation is very confusing about this - it might also be generated when setting up a Chromium OS from scratch (my thread on this is halted for now, but I have all the setup foundation to build)...
What I gather is that vmlinuz.A is the kernel, but for chromebooks, the kernel is also contained in the blob which is in partition /dev/sdx2 labelled KERN-A -
so I need to produce an image kern-a.bin which I then flash to /dev/sdx2 - with FydeOS 5.3.1 the KERN-A blob contains Kernel 4.14.67, with ATB v72 Kernel 4.14.83...
the dilemma is that KERN-A size for FydeOS 5.3.1 is 16 MB & for ATB v72 it is 64MB, so I cannot flash the latter onto the former.
I need to find out how to create a 64MB image for KERN-A from a given kernel, such as ATB v72 or my compiled one which is 4.14.96 -
Anyone who has a clue, & better has the correct instructions to do so, please help...
Actually, you don't need to: you can use the compressed image with GRUB2, and it will boot using your system and state (data) partition.
lfom said:
Actually, you don't need to: you can use the compressed image with GRUB2, and it will boot using your system and state (data) partition.
Click to expand...
Click to collapse
Using a modified FydeOS 5.3.1 as base, no matter I change the kernel, it boots its own 4.14.67 kernel on Acer CB C720P - however using non chromebook devices such as macbook, allow me to boot vmlinuz.A & its associated /lib/modules/...
Care to share how to use grub2 to override the above?
Hi everyone
To introduce , i wanted to build kali nethunter for my device but he is not builded so i go and try to build a kernel following the official github kali nethunter for porting device (link2),[a video tutorial](link 2) where i did same as the guy but at the end i got error and i also searched on stackoverflow for crosscompiler issue and resolved it. But now i have trouble that i doesn't know how to do with when building my kernel
Device: xiaomi redmi note 3 snapdragon version
codename: kenzo
I didn't find original kernel for kenzo so i took one custom kernel on xda forum :[github](link 2) and a [toolchain for 64 devices](link 3)
so i started building until modifing .config w/o trouble
Code:
make menuconfig
Code:
make
Code:
cp .config arch/arm/configs/kali_defconfig
Code:
make clean && make mrproper
Code:
mkdir ../out1
until now this was good but when i start to build defconfig
Code:
make O=../out1 kali1_defconfig
my .config was done but i get [error message](link 4) in
When i want to make my kernel this don't make it : [error](link 5)
Code:
make O=../out1 -j2
w/o using -j [error](link 6)
Code:
make O=../out1
[Makefile](link 7) [confdata.c](link 8)
I tried to give you all i can so sorry for using a lot of pastebin
Hope someone can try to help me ty
link 1 : youtube. com/ watch?v=6-oKLou69WM
link 2 : github. com /GuneetAtwal/kenzo
link 3 : android.googlesource. com /platform/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9
link 4 : pastebin. com /M0TtmMzS
link 5 : pastebin. com /2azLQgzN
link 6 : pastebin. com /J8JFGW41
link 7 : pastebin. com /s3Dycgm4
link 8 : pastebin. com /8FQJFWxJ
Little info on what this all is.
If you are expecting a working kernel or rom after reading this all. That is not what this thread is.
What this all is intended for is material to help with getting you to the next step, if you are at a road block.
What will you gain from all this?
A fair bit of knowledge on working with msm-4.14 kernel and maybe others.
Make sure to have your search bar ready with a piece of the issue (key word of the error will do) and go through the hidden tabs until you pick up your error, with the browser search function (ctrl+f on most browsers)
Spoiler: OLD info
Method I used for building the kernel
1. Dowload AOSP common-kernel-4.14 through git
OR
ANDROID 10 Specific
refs/heads/q-common-android-4.14 - kernel/manifest - Git at Google
2. Download Xiaomi TUCANA-Q-SOURCE for android 10 source code from MiCode git.
ALL VARIANTS
GitHub - MiCode/Xiaomi_Kernel_OpenSource: Xiaomi Mobile Phone Kernel OpenSource
Xiaomi Mobile Phone Kernel OpenSource. Contribute to MiCode/Xiaomi_Kernel_OpenSource development by creating an account on GitHub.
github.com
**OR**
TUCANA Android 10
GitHub - MiCode/Xiaomi_Kernel_OpenSource at tucana-q-oss
Xiaomi Mobile Phone Kernel OpenSource. Contribute to MiCode/Xiaomi_Kernel_OpenSource development by creating an account on GitHub.
github.com
3. extract if you did not use git to download sources.
You will need AOSP 10 R29 downloaded. R29 Matches Stock MIUI 12.0.4
4. mkdir kernelbuild (can be what ever) then extract both AOSP KERNEL (git) ,build, common, kernel, prebuilts, master-prebuilts folders to the kernelbuild folder.
5. just use TUCANA source and don't merge with AOSP common source. Don't delete common either
4. copy AOSP kernel folder (contains 4.14) to TUCANA source and merge (Still not sure if you need to do this)
5. make sure you have AOSP-10 from git already. If not, repo it.
6. Go to AOSP-10 REPO>prebuilts>gcc>linux-x86>aarch64 location and get (aarch64-linux-android-4.9) folder and copy it to root Tucana source files folder make a directory call toolchain TUCANA source. in prebuilts>gcc>linux-x86>aarch64 folder replace and merge all
7. Not sure if this is outdated but got the export info from MSM section at https://github.com/MiCode/Xiaomi_Kernel_OpenSource/wiki/How-to-compile-kernel-standalone
e.g.
export CROSS_COMPILE=/<toolchain-location/prebuilts>GCC>/bin/aarch64-linux-android-
mkdir out
cd out
Taken from mi code section for MSM 4.14
export ARCH=arm64
export SUBARCH=arm64
export DTC_EXT=dtc
Set CONFIG_BUILD_ARM64_DT_OVERLAY=y
(does not work from what i can tell. Have to enable using menuconfig)
if not inside the "out" folder use O=out on next command, will also have to type cd ../ to go back in source if using this command.
(-jN (N is for a number))
make -jN tucana_user_defconfig
make menuconfig (configure config to your liking)
use save button (highlight save and press enter, can use arrow key right and left as well)
make -jN
Or it could be
make -jN ARCH=arm64
After doing all this, for me the build fails. Its driving me insane!
I think I also was using the wrong AOSP kernel, was using common-4.14 and now using this one
refs/heads/q-common-android-4.14 - kernel/manifest - Git at Google
guessing the Q is for Android 10
MY DEMON!!!!!
1615228944041.png
i think it is not working because i am using Android 10 R41 and should be using Android 10 R29. Testing it now.
Alright, went from ubuntu version 20 to 18 (Bionic is more reliable and easier to set up).
Had 20, due to bionic auto updated (my own fault).
Version 20 does work with some modding and adding bionic to the repo's of ubuntu.
I tried using AOSP toolchain and also Qualcomm's (QQ-LLV) LLV toolchain 8.0 for clang.
The part from MiCode wiki on github is saying to use CLANG_Triple with aarch64-linux-gnu which I can not find this file anywhere.
Not in QQ-LLV, AOSP-10-R29 aarch64 that is located in prebuilts > gcc
so the main problem I am getting is the cpu timer failing, during the build process. I try to modify the config by typing "make menuconfig" and then change the cpu govern type from performance to ondemand and saving it as .config (not sure if it is supposed to be the same name as the tucana-user-defconfig, let me know if this is the problem)
the AOSP android version I am using is R29. The AOSP kernel I am using is common-4.14
none of these files have aarch64-linux-gnu.
I am starting to slowly give up on this whole thing. spent 7 days just to get a cpu timer problem during build.
Oh and the source i am using is from Micode github under tucana for android 10 (Q)
Just sprung up an idea. I think i am supposed to first use Qualcomms LLVM toolchain 8.0 with the Mi source code package to make up the files needed in order to use anything from AOSP. please let me know if this is correct.
No matter what guide i find for it, it shows CLANG_TRIPLE=aarch64-linux-gnu- and every time. it just can not find it. no idea how to get this.
it work in clang triple for -gnu
AOSP-Q-Kernel-4.14.117
refs/heads/q-common-android-4.14 - kernel/manifest - Git at Google
Spoiler: Kbuild config
ARCH=arm64
SUBARCH=arm64
BRANCH=android-4.14
CLANG_TRIPLE=aarch64-linux-gnu-
CROSS_COMPILE=aarch64-linux-android-
DEFCONFIG=tucana_user_defconfig
KERNEL_DIR=xsource
DTC_EXT=dtc
DTS_EXT=dts
CC=clang
LZ4_RAMDISK=1
POST_DEFCONFIG_CMDS=""
EXTRA_CMDS=''
CLANG_PREBUILT_BIN=prebuilts-master/clang/host/linux-x86/clang-r353983c/bin
LINUX_GCC_CROSS_COMPILE_PREBUILTS_BIN=prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin
REAL_CC=prebuilts/ndk/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang
BUILDTOOLS_PREBUILT_BIN=build/build-tools/path/linux-x86
FILES="
O=/out
arch/arm64/boot/Image.gz
vmlinux
System.map
"
STOP_SHIP_TRACEPRINTK=1
OMFG!!!!!!! VICTORY!!!!!!!!
I think.
1615380568663.png
MAYBE!!!!!
1615380671546.png
Spoiler: New info
Alright Managed to fix most of the problems in the OLD area, mostly due to path issues, always check your paths (PATH).
Spoiler: build.config
ARCH=arm64
SUBARCH=arm64
BRANCH=K4.14Q
CLANG_TRIPLE=aarch64-linux-gnu-
CROSS_COMPILE=~/android/xkernel/tsource/toolchains/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin/aarch64-linux-android-
CROSS_COMPILE_ARM32=~/android/xkernel/tsource/toolchains/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin/arm-linux-androideabi-
KBUILD_DEFCONFIG=~/android/xkernel/tsource/arch/arm64/configs/tucana_user_defconfig
DEFCONFIG=tucana_user_defconfig
POST_DEFCONFIG_CMDS="check_defconfig"
DTC_EXT=dtc
DTC_PREBUILTS_BIN=/scripts/dtc
KBUILD_OUTPUT=out
HOSTCC=gcc
CC=clang
AS=clang
AR=ar
CLANG_PREBUILT_BIN=/toolchains/clang/host/linux-x86/clang-r353983c/bin
BUILDTOOLS_PREBUILT_BIN=/toolchains/build-tools/linux-x86/bin
FILES="
arch/arm64/boot/Image.gz
vmlinux
System.map
"
Don't know if most of what is in there is needed or if everything I need is there but that is what i have so far.
One thing that i know always does not work is the check_defconfig. fails to match every time.
Tried using ld.ldd and it just keeps saying the vmlinux file size is too large. I have not given up and can say i have learned a lot.
Will update more as I go.
Spoiler: Links
Toolchain and kbuild config help
Hello I have been trying to extract the kernel from Tucana android 10 source. I would like to know if anyone has a working config to be able to build up the kernel. I have a config but it does not extract everything, well I don't think it does...
forum.xda-developers.com
[SOLVED] dts not found
Hello. Been working on how to get a device kernel from source for the Mi note 10 pro (Mi CC9 Pro). I have gotten up to the point where it builds but fails due to dts folder is not found. I need an example of what goes in DTC_EXT= All I see on...
forum.xda-developers.com
Spoiler: UPDATE 2022 WORKING BUILD WITH ISSUES
Alright I decided to give it another shot in 2022 because of getting replies.
1. Get your source from device brand. (https://github.com/MiCode/Xiaomi_Kernel_OpenSource)
2. Use Mi code wiki to learn how to build (https://github.com/MiCode/Xiaomi_Kernel_OpenSource/wiki)
3. Go right hand side for standalone kernel. Because the how to section just waste hours on end with nothing built and Soong and clang errors.
4. Follow msm-4.14 for a guide. (https://github.com/MiCode/Xiaomi_Kernel_OpenSource/wiki/How-to-compile-kernel-standalone)
5. Make sure to have dtc binary file from aosp source (https://android.googlesource.com/platform/prebuilts/misc/) choose your android version. Check device for android version. Can use CPU ID in playstore under System, section API LEVEL.
6. Get llvm Snapdragon from Qualcomm. (https://developer.qualcomm.com/software/snapdragon-llvm-compiler-android) I used both but you should be able to use 8.0. both have off same results during the build process.
7. Get Aosp gcc
(https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/)
8. Put LLVM Snapdragon toolchain and AOSP gcc into a folder in root of kernel folder called toolchain.
9. Run the script mi provided. Change the 6.0 on the last 2 make commands to 8.0 unless you are already using 6.0, then you can leave it at 6.0.
10. Build.
11. Get a boot.img extractor from github or on these forums. I can't recommend one yet, due to not having full success. Can search boot.img extractor or boot.img unpacked as seperate search terms.
**Recommendation**
Boot and Recovery
GitHub - xiaolu/mkbootimg_tools: Unpack and repack boot.img,support dtb(dt.img).
Unpack and repack boot.img,support dtb(dt.img). Contribute to xiaolu/mkbootimg_tools development by creating an account on GitHub.
github.com
OR
Android Kitchen
GitHub - osm0sis/Android-Image-Kitchen: Automated scripts to unpack/repack Android kernel/recovery images + ramdisks
Automated scripts to unpack/repack Android kernel/recovery images + ramdisks - GitHub - osm0sis/Android-Image-Kitchen: Automated scripts to unpack/repack Android kernel/recovery images + ramdisks
github.com
OR
Android Kitchen
GitHub - cfig/Android_boot_image_editor: Parsing and re-packing Android boot.img/vbmeta.img/payload.bin, supporting Android 13
Parsing and re-packing Android boot.img/vbmeta.img/payload.bin, supporting Android 13 - GitHub - cfig/Android_boot_image_editor: Parsing and re-packing Android boot.img/vbmeta.img/payload.bin, supp...
github.com
12. Extract boot.img from stock rom.
13. Put image.gz.dtb in unpacked boot image. Delete original file and rename new one as the same name.
14. Repack boot.img and either upload using fastboot or use twrp to install the boot.img
The problem I have at the moment is I am unable to use touch input at all. It boots loads system but can't touch anything. No input at all. Hardware buttons work though.
Might be solution on 3rd post to no touch input. https://forum.xda-developers.com/t/reference-how-to-compile-an-android-kernel.3627297/page-43
VMLINUX FIX: HERE
Hi @Squida, I've also been on this road, I just hit a milestone when I actually have something booting (but not much working), see the thread I started if it works for you too
b100dian said:
Hi @Squida, I've also been on this road, I just hit a milestone when I actually have something booting (but not much working), see the thread I started if it works for you too
Click to expand...
Click to collapse
Spoiler: Reply
Hey mate, thanks for the reply. Could you put a link to your thread so I can check it out.
Found it: https://forum.xda-developers.com/t/building-lineageos-17-1-from-source.4245417/
I also have posted on Qualcomm forums.
LINK: https://developer.qualcomm.com/forum/qdn-forums/software/snapdragon-llvm-compiler-android/68403
I found out qcomm has its own builder for msm devices. It's known as QAEP. I have been trying to build the sm6150 with Tucana defconfig. The thread above is the issue I have in trying to build it. No idea how to get "SDClang" so as soon as I work that out. It should build with QAEP instead of using AOSP.
There's a commit where I add SDCLANG support (but I don't think that's needed, is backed out atm) https://github.com/alibei/android_d...mmit/efbb3c66aa0d385e58052675402489c13392576e )There's a similar commit in the kernel.
You need to register to Qualcomm to download it, and unzip it in kernel/toolchains, just as https://github.com/MiCode/Xiaomi_Kernel_OpenSource/wiki/How-to-compile-kernel-standalone says under "You must get llvm clang from qcom". You just have to get the 8.x version.
Some snippets for the commands I've tried https://gist.github.com/b100dian/40c8dbe746ff181aff71ee10a75a5f3c
Spoiler: Reply
Yeah I am not trying to use Lineage sources. so far steps I have taken are as follows.
1. Mi source code download of Tucana android 10, data source, wifi source, audio source.
2. Extracted Mi tucana source in a directory (also tried the git way to update CAF tags).
3. using AOSP as the source for build. I tried using Clang with cross compile with gcc (GNU)
4. with using all AOSP toolchains so clang and gcc from AOSP and then using LLVM clang as reall cc, due to it does not contain clang itself.
all that for kernel and it builds but I feel its missing drivers. due to the warnings that Qcom gives. also got wifi modules installed but not audio. Audio source is a little different then the wifi source.
For the proprietary binaries. I used lineage Extract script with the lineage 17.1 tucana proprietary-files.txt list.
extraction worked on miui_TUCANAGlobal_V12.0.4.0.QFDMIXM_be49be8fa0_10.0.zip.
but of course the device tree is missing, found the platform sm6150 device tree on QAEP. so now trying to use QAEP to build not only the kernel but the rom as well using QAEP instead of AOSP. think we need to use QAEP, then using the files built. can then move over to aosp for upgrading, etc.
Could you explain this error considering you managed to get SDClang working. it may solve my problem. error provided below.
Spoiler: sdclang error
I have a SDCLANG_PATH set in BoardConfig.mk in the commit I pointed to earlier.
Basically is from a toolchains folder creaded with `tar -xvzf snapdragon-llvm-8.0.6-linux64.tar.gz` in the kernel/xiaomi/tucana folder
b100dian said:
I have a SDCLANG_PATH set in BoardConfig.mk in the commit I pointed to earlier.
Basically is from a toolchains folder creaded with `tar -xvzf snapdragon-llvm-8.0.6-linux64.tar.gz` in the kernel/xiaomi/tucana folder
Click to expand...
Click to collapse
what i have setup in boardconfig is as follows.
ifneq ($(HOST_OS),linux)
SDCLANG := true
SDCLANG_PATH := toolchain/ndk/android-ndk-r22/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin
SDCLANG_LTO_DEFS := device/qcom/common/sdllvm-lto-defs.mk
endif
this has been added just above wifi. still fails with same error.
Spoiler: Boardconfig.mk
Image:
I think I just solved it, so i noticed you talk about Boardconfig.mk and when i checked out your git code, it had in the same config the target. well QAEP is a little different it not only has a Boardconfig file but also a AndroidBoard config file. so i think i am supposed to be adding it in there instead of Boardconfig. testing it now.
Spoiler: AndroidBoard.mk
Something else i also noticed that is not working.
Spoiler: .ko files missing
And can confirm above configuration changes are not working, not sure where it is located to tell it the path.
I cannot speak for QAEP (I barely began reading about lineage but in my case I don't have out/target/product/sm6150 at all, only out/target/product/tucana, which seems to be the name of the kernel (or device or vendor). Do you also have other repos pulled in that contain kernel named sm6150? Like https://github.com/LineageOS/android_kernel_xiaomi_sm6150 Maybe you should not have both
b100dian said:
I cannot speak for QAEP (I barely began reading about lineage but in my case I don't have out/target/product/sm6150 at all, only out/target/product/tucana, which seems to be the name of the kernel (or device or vendor). Do you also have other repos pulled in that contain kernel named sm6150? Like https://github.com/LineageOS/android_kernel_xiaomi_sm6150 Maybe you should not have both
Click to expand...
Click to collapse
Spoiler: b100dian reply
So where yours is out/target/product/tucana, the one for QAEP is SM6150 which is in the same location. so I think maybe on the right track here. also thanks for the help for llvm dragon, had to set bin path, for it to work. but i have not gone a built the rom yet, doing the kernel build first now.
Instead of relying on QAEP builder. I went and got kernel/build from code-aurora for android 10 r40 and ended up getting misc linux folder in kernel also gcc and build tools from the device tag release repo on codeaurora.
Usually i was using AOSP but now switched over to all QAEP tools and sources.
I am testing the Micode Audio and wifi source. got wifi working during kernel build by adding this below.
EXT_MODULES="
mods/wlan/qcacld-3.0
"
IN_KERNEL_MODULES=1
I believe that installs the wifi drivers for the device as a module. but building with clang, gcc and gnu as clang tripple this is all related to only kernel building.
Spoiler: Update
Done an overhaul on everything for the kernel. decided to switch from QAEP to AOSP.
Reason for the switch, I noticed with AOSP I am able to not only download the kernel/msm4.14 common folder but it also downloads build, prebuilts, prebuilts-master folders with everything included.
Differences aside from what I mentioned above.
AOSP common-4.14 comes with up to date builder, gcc, clang, etc. But when I download the release tag for example, LA.UM.9.1.r1-06700-SMxxx0.0 at codeaurora on the otherhand, it only downloads the common folder and no extra folders for it.
So then in turn. you have to go on codeaurora and download by using git clone, the build folder for what ever android version it is. for example, I had android version 29 (r29) though it does not exist in the branch list on codeaurora so i went with 28.
I should also note that making the change from QAEP to AOSP I started again with the kernel source.
With making the change, I have noticed improvements right away, but also I noticed that having CC=clang, the build would not work untill I also put in HOSTCC=gcc. guessing its to do with the AOSP version being a little different. still using QualComms LLVM clang compiler for everything clang related.
Spoiler: Problems I have so far.
***Error-1***
Have it building but get this spam, been trying to solve it.
***Error-2***
This error pops up when using LD=ld.lld
this is what is set in the build.config
***CONFIG-1***
LD=ld.lld
LD_LIBRARY_PATH=toolchain/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/lib/clang/8.0.6/lib/linux/aarch64:$LD_LIBRARY_PATH
export O=out/android-4.14 LD_LIBRARY_PATH
**RESULTS**
***CONFIG-2***
LD=ld.lld
HOSTLDFLAGS="-fuse-ld=lld" (added it so i can modify)
LD_LIBRARY_PATH=toolchain/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/lib/clang/8.0.6/lib:$LD_LIBRARY_PATH
export O=out/android-4.14 LD_LIBRARY_PATH
**RESULTS**
Same as above.
Spoiler: Config commands
This right here may have just solved all my problems. can confirm in all my config tries. Never once thought to put aarch64-linux-android before all variable values.
SOURCE: https://developer.android.com/ndk/guides/standalone_toolchain
# Tell configure what tools to use.
target_host=aarch64-linux-android
export AR=$target_host-ar
export AS=$target_host-clang
export CC=$target_host-clang
export CXX=$target_host-clang++
export LD=$target_host-ld
export STRIP=$target_host-strip
Also discovered that LD=ld.lld is for clang compiler. could be why the above Problem is happening.
This is using the toolschains without NDK, may have to put qualcomm and gcc in NDK and path them.
So after trying to downgrade to android 10 from 11, had no luck in doing so.
So now sticking with android 11 and I can safely say that I have reached a mile stone.
I ended up downloading android 11 R30 and using all the tool chains from AOSP while still using Qualcomm's LLVM compiler 8.0.6 as REAL_CC. But the clang pre-built path is clang from AOSP. Just trying to work out how to include the Xiaomi audio source. Unable to work out where to put it. Still testing, have not given up.
Hi @Squida, where is `mods/wlan/qcacld-3.0` from, the qcom talos referenced above in BoardConfig.mk?
Do you have an exact link to the repo?
In the meantime I can confirm I am missing something form the kernel. If I build everything myself and replace _just_ the kernel (with my dtb appended etc) it has sound/wireless, so knowing what wifi / audio module to link in seems the way to go.
b100dian said:
Hi @Squida, where is `mods/wlan/qcacld-3.0` from, the qcom talos referenced above in BoardConfig.mk?
Do you have an exact link to the repo?
In the meantime I can confirm I am missing something form the kernel. If I build everything myself and replace _just_ the kernel (with my dtb appended etc) it has sound/wireless, so knowing what wifi / audio module to link in seems the way to go.
Click to expand...
Click to collapse
Spoiler: Reply
No problem at all, anything to get this build done lol.
All links are for Android 11
If you want Android 10, replace pheonix-r-oss with tucana-q-oss
**Example for Android 10**
GitHub - MiCode/Xiaomi_Kernel_OpenSource at tucana-q-oss
Xiaomi Mobile Phone Kernel OpenSource. Contribute to MiCode/Xiaomi_Kernel_OpenSource development by creating an account on GitHub.
github.com
******ANDROID 11******
***SOURCE***
GitHub - MiCode/Xiaomi_Kernel_OpenSource at phoenix-r-oss
Xiaomi Mobile Phone Kernel OpenSource. Contribute to MiCode/Xiaomi_Kernel_OpenSource development by creating an account on GitHub.
github.com
***DATA***
GitHub - MiCode/vendor_qcom_opensource_data-kernel at phoenix-r-oss
xiaomi opensource for data-kernel. Contribute to MiCode/vendor_qcom_opensource_data-kernel development by creating an account on GitHub.
github.com
***AUDIO***
GitHub - MiCode/vendor_qcom_opensource_audio-kernel at phoenix-r-oss
Contribute to MiCode/vendor_qcom_opensource_audio-kernel development by creating an account on GitHub.
github.com
***WIFI (mods/wlan/qcacld-3.0)***
MiCode/vendor_qcom_opensource_wlan
Contribute to MiCode/vendor_qcom_opensource_wlan development by creating an account on GitHub.
github.com
Thanks, I think I get now what you're saying with 'talos', I poked at those repos now.
It seems to me the modules are built out of tree, as they don't appear in the extracted kernel config. (See https://github.com/b100dian/Xiaomi_Kernel_OpenSource/commit/0f4b062ef806aaad6ad02e2efd87809e8b7250c6).
Also, having dlkm from https://source.codeaurora.org/quic/la/device/qcom/common/tree/?h=LA.UM.8.9.r1-03800-sm6150.0 does not seem to help automate the build, I get all sorts of errors when AndroidKernelModule.mk is included and it includes back the AndroidKernel.mk..
Getting these into drivers/staging seems easier at first but..for now, I've only managed to build wlan.ko but that still errors out with `wlan: disagrees about version of symbol module_layout` :-S
b100dian said:
Spoiler: Reply
Thanks, I think I get now what you're saying with 'talos', I poked at those repos now.
It seems to me the modules are built out of tree, as they don't appear in the extracted kernel config. (See https://github.com/b100dian/Xiaomi_Kernel_OpenSource/commit/0f4b062ef806aaad6ad02e2efd87809e8b7250c6).
Also, having dlkm from https://source.codeaurora.org/quic/la/device/qcom/common/tree/?h=LA.UM.8.9.r1-03800-sm6150.0 does not seem to help automate the build, I get all sorts of errors when AndroidKernelModule.mk is included and it includes back the AndroidKernel.mk..
Getting these into drivers/staging seems easier at first but..for now, I've only managed to build wlan.ko but that still errors out with `wlan: disagrees about version of symbol module_layout` :-S
Click to expand...
Click to collapse
Spoiler: Reply
It seems you are having a similar issue.
When doing everything I have done so far, The biggest wall for me is getting <LD=LD.lld> to work properly, I either get clang error with a file having a linking problem with using clang from QQ llvm compiler. And when I switch over to using AOSP's 4.14 stable kernel prebuilts-master clang. It works but then another error to do with CPU timer pops up. Which was originally because of using gcc instead of clang in the CC=clang environment variable, or removing it out of the build.config file.
i myself did a major overhaul of Ubuntu and have upgraded to LTS 20.04 from 18 bionic.
Noticed some things right away.
When you install gcc-multilib and g++-multilib. You get version 9 instead of you being on Ubuntu bionic and getting version 7. Also faster in general with general use of the operating system using Oracle vmbox. On windows 10.
Apart from linking errors, kernel is buding but only with not having LD=LD.lld in the build.config file.
Spoiler: wifi-info
And for the wifi qcom. Mods folder I added manually. Inside the devices source folder. Then copied WLAN sources inside and renamed the folder to just WLAN.
Spoiler: where I am at now
I have got both android 11 and android 10 kernel and ROMs. Also went and downloaded the kernel for coral. That's how I found out about the wifi and how I got a lot of the settings for the build.config.
I am gonna have a break on this whole thing, gotten to a point where with android 11 Xiaomi Phoenix. I get one error with a file in kbuild, using the same config settings for android 10. I get none. Apart from the qcom space which is either DTC not working or failing at some point. Not sure if we need to use LD=LD.lld I have been reading it is not needed due to the builder choosing the correct one for you.
Overall though it builds. Just the spam for qcom warnings I just can not get rid of and it's to do with the sensors.
If anyone has information on what causes the qcom spam. Please let me know, thank you.
Plus this all started with vmlinux not working right. Something to do with channel scratch. Can not fix it.
Spoiler: My Thoughts
I have just found this exploring info on the rom.
Build AOSP with LineageOS device tree
My device (Xiaomi Redmi Note 5, whyred) have official LineageOS support and therefore there is device tree and kernel. I want to build AOSP without any modifications or tweaks. How can I use (or po...
stackoverflow.com
Key part taken from above link.
**Its hard to build pure AOSP than other ROMs. While building a custom ROM a lot of components wont work**
I think for our devices we have no choice but to use QAEP instead of AOSP, it explains why it fails to build properly. we first need a full working rom and then I think we can move over to AOSP and modify lets say, coral device and match it for our device. Though I think it is just easier using QAEP and why QAEP exists in the first place.
When I used QAEP, I got the SDCLANG problem, the build process according to the Micode git wiki, section "How to use" explains on how to configure and by the looks of the guide. It seems simple as cooking a boiled egg with only a couple of modifications to a device tree and adding your kernel to the kernel folder. It is supposed to build. I myself get SDClang path problems when using QAEP though.
I will go back to QAEP but i will be using all android 11 sources. due to unable to downgrade my phone from android 11 to 10. Having issues at the moment with TWRP installing. Shows my folder structure folder and file names. all random characters.
I will update as i get further in building the kernel and rom
Spoiler: Thoughts2
So I am now merging the built kernel into the kernel directory of AOSP 10. Made a folder called common and moved the built kernel into it. Also made a folder called prebuilts and put in all the .gz into it under a folder called 4.14. discovered that the AOSP 10 picked up the common folder with the built kernel but is having errors with Android.bp, will update as I find out more. Right now, just tinkering.
This is the issue when you have the build tools outside the source folder.
Could be a path I have not set properly. once I removed the command from POST_DEFCONFIG_CMDS="" it is now building.
Couple of things to note.
REAL_CC= I believe has now changed to HOSTCC=. Command is not found in any files in the build directory of the AOSP 4.14 STABLE Kernel.
QualComms LLVM 8.0.6 Compiler does not contain clang files. you still need to either download it, or just use the one in AOSP 4.14 Stable kernel, plus the clang version in AOSP stable kernel is a later version and also contains clang version for android 10+ instead of android 9.
New error I get after changes mentioned above are made.
Not sure where to put -fPIC to make the command function.
Fixed it, was a pathing issue with folders.
But now I am stuck now on finding out the directory for LD_LIBRARY_PATH=
Code:
DTC arch/arm64/boot/dts/qcom/apq8016-sbc.dtb
dtc: error while loading shared libraries: /home/avm/dev/source/kernel/prebuilts-master/ndk/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/lib/libc++.so: file too short
make[4]: *** [scripts/Makefile.lib:325: arch/arm64/boot/dts/qcom/apq8016-sbc.dtb] Error 127
make[3]: *** [/home/avm/dev/source/kernel/mi10/scripts/Makefile.build:676: arch/arm64/boot/dts/qcom] Error 2
make[2]: *** [arch/arm64/Makefile:187: dtbs] Error 2
make[2]: *** Waiting for unfinished jobs....
CC lib/rhashtable.o
Spoiler: Extra Discovered Info
${ROOT_DIR} is used. With it and you don't need to export it. Builder already knows what it is. so you use it like this for example
************************************
CC=${ROOT_DIR}/path-to-QQ_LLVM_COMPILER clang binary file, located in bin folder.
Also do this for ld.lld
LD=${ROOT_DIR}/path-to-ld.lld
************************************
Found another odd thing that happens
Having CC=PATH-TO-QQ-llvm-compiler/clang with CC_PREBUILT_BIN as aosp's location and HOSTCC=clang
It builds.
But with LD=
It fails.
I try using QQ-LLVM-Compiler as CC_Prebuilt_bin=
Fails.
So far I have managed to get to this point.
And can safely say that using QualComs LLVM 8.0.6 compiler does not work with android 10 so stick with AOSP's prebuilt clang either in Kernel stable git or AOSP android-Number-Revision
Again, REAL_CC does not work, nor show up in the enviroment as a used command.
These below all do.
<CC=>, <HOSTCC=>, <HOSTLDFLAGS=-fuse-ld=lld>, <NM=llvm-nm>, <OBJCOPY=llvm-objcopy>, <HOSTLD=>, <LD=>,
HOSTCC= will use what is put in CLANG_PREBUILT_BIN=
CC= you are able to add QQ-LLVM-COMPILER's clang and it will run.
Proof of QQ-LLVM-COMPILER failing, when using it for <CLANG_PREBUILT_BIN=>
Instead of using AOSP's clang.
So far that is all I have seen pop up.
All these changes were done with the Kernel AOSP-COMMON-4.14-STABLE build, kernel, prebuilts, prebuilts-master folders and then with the devices source in a folder with a build.config file in the same location as the AOSP-kernel build folders. this all seems to work, Only thing I lose out on, is using POST_DEFCONFIG_CMDS="" to use make menuconfig.
Reason that make does not work is because according to AOSP coral's kernel you have another build.config file which is with the build folder, prebuilts, master-prebuilts folders which connects to the main build.config.common and clang config that has the extra commands. So in turn I do not think we use LD=ld.lld
We let the builder choose it for us.
95% sure that the DTC is what is causing qcom to have that warning spam.
Device tree compiler (DTC) linking seems to be the full cause of all the issues.
Okay to prevent fixdep.c:105:10: error all you need to do is have HOSTCC=gcc and it works. so then in turn. You can now use QQ LLVM Clang compiler in CLANG_PREBUILT_BIN= instead of AOSP.
Spoiler: Helpful Links
GitHub - nathanchance/android-kernel-clang: Information on compiling Android kernels with Clang
Information on compiling Android kernels with Clang - GitHub - nathanchance/android-kernel-clang: Information on compiling Android kernels with Clang
github.com
Spoiler: Still going
Still going at it, manage to find out a couple of things.
If your build is failing due to files missing, check your build directory and make sure the files in the path folder and in Linux-x86>bin folders are indeed "symlinked" something I was unaware of with Linux due to me being a windows user. none of the icons can have an X on it. it means its broken. this solved a ton of my issues during Rom building.
the build folder itself is a symlink folder. (Not all files)
Just locate and put files in correct places. so instead of having a folder called toolchains, like we are told on the Micode wiki, etc. If you use the AOSP-kernel build, prebuilts, prebuilts-master. which contains all kernel and build tools. and are already linked. so then you just add QQ-LLVM-Compiler to either prebuilts or pre-builts master and link it in the config.
If you do all this correctly and having all files in the proper location, you should not have any more build errors in regards to missing files.
Summed up
Build directory contains symlinks to binary files which are located in prebuilts and prebuilts-master. you need all 3 folders in the device source root directory.
Inside the build directory you will find a file called build.config, its not a config. its a symlink for one. Rename it to the build.config.Aarch64 for example, or what ever your config is called located in your root devices source directory. this solved a **** load of my problems.
The problem I have now is actually got to do with Repo, AOSP, QAEP. found out using the manual install method of repo makes it so you can download off google when you have the pgp key installed. but having it installed this way, stuffs up gpg for QAEP. I am looking for a way to merge all PGP keys in one location.
and to get QAEP downloading without showing this error.
All you have to do is remove the manual repo you installed and then also deleting the .repoconfig folder and .gnupg folder.
Then install repo with <sudo snap install git-repo>
it will work, you will get a public key under john doe and it will use CAF as well. problem is. You lose out on public key to AOSP and also the repo that is inside the AOSP folder needs to be replaced. trying it just causes errors though.
What happens when you do above by removing the manual repo and adding the snap git-repo
So why does it matter that I need both public keys to work?
Because I am using AOSP and also QAEP for testing.
I'll be sticking with the snap instal git-repo for now. i'll have to work out how to add the key from AOSP.
**Error when going back to AOSP**
**what happens when you delete the repo folder inside .repo of AOSP**
Only way I got it to work is by removing the snap install version of repo and install the manual way.
But then I lose out on CAF.
Think i may have worked out what is happening with the keys. 2 folders are being made. .repoconfig and .gnupg.
.gnupg only gets created when using AOSP pgp key.
and there is a gnupg folder in .repoconfig, might be a command to merge the files.
this command below makes it so you can change the name.
--config-name
put it at the end of the repo init command.
from what I understand for the tucana_user_defconfig.
#CONFIG_BUILD_ARM64_DT_OVERLAY does not need to be set. for mi 9, yes. not for mi note 10. Just pulled the defconfig from my device using the command below.
**EXTRACT DEFCONFIG FROM DEVICE**
MSM devices
CMD> adb pull /proc/config.gz
files appears as config.gz in same directory
I should also note that I ran 2 builds one with DT_OVERLAY=Y and then one =N
the outcome was with the change of CONFIG_BUILD_ARM64_DT_OVERLAY=n
That is not how it is setup, the proper way to disable it is as follows.
#CONFIG_BUILD_ARM64_DT_OVERLAY is not set
I got a bigger file size for Image.gz-dtb. instead of under 20MB
Have it on so you can extract DTBO and dtb files. having it off stops it from extracting.
THIS error or what ever the hell it is, is driving me insane.
My understanding is that DTBO (overlay), when enabled, ends up in dtbo.img, and when disabled, probably ends up in dtb which is appened to the kernel Image.gz.
The gsi_write_channel_scratch error is... maybe this helps? https://github.com/ClangBuiltLinux/linux/issues/931#issuecomment-599681910
Spoiler: reply
Btw, thank you @Squida for pointing me out the correct repos for wifi and audio, I managed to compile both into my build and the audio one inline (the wifi needs to be insmod'ed).
b100dian said:
My understanding is that DTBO (overlay), when enabled, ends up in dtbo.img, and when disabled, probably ends up in dtb which is appened to the kernel Image.gz.
The gsi_write_channel_scratch error is... maybe this helps? https://github.com/ClangBuiltLinux/linux/issues/931#issuecomment-599681910
Spoiler: reply
Btw, thank you @Squida for pointing me out the correct repos for wifi and audio, I managed to compile both into my build and the audio one inline (the wifi needs to be insmod'ed).
Click to expand...
Click to collapse
Spoiler: Reply
You sir, are a Legend!
Okay So if I am reading that right.
If you mean DT_OVERLAY=y in defconfig, makes it so I can get the dtbo.img file. I tried adding the dtbo.img to the list of files for extraction and with it on, no dtbo image is made nor found, with it off. still the same thing but unable to extract any dtb or dtbo images from the boot directory. have to have it on.
dtbo.img just does not want to extract for me.
Again, using the clang config and not the mi 9 config used on Micode wiki.
the Micode Wiki settings do not work correctly. you lose out on about a GB of stuff and I know its a DTC issue I have been having due to the Qcom spam.
I have reduced the space though so now you can actually scroll through the whole build process, including the spam.
I will definitely be looking into that solution for channel scratch.
Also if you could please show me the config for the SDClang settings you put it to get it to detect. I tried looking over your git page and failed to understand why you have it in boardconfig.mk and if lineage has other files for the device.
Reason being is because on QAEP it talks about SDclang-3.8 and i have no idea where it is.. Supposed to be in QQ-LLVM-compiler and you copy it to the prebuilts folders, it just does not exist to do that.
the major problem I have is building the device tree when building the kernel. from my understanding, Xiaomi have set it up so DTC creates a Device tree for you. well that is the part that is failing and the whole reason why I can not make a rom.
My guess because you are using lineage sources, everything I am talking about. they already did for you. I am trying to do it all manually. learning purposes.
Okay so it turns out, we need LD=ld.lld
Though we must modify files, this is ridiculous just to get your own rom for xiaomi devices.
This thread below may have the solution for me and an easier one then having to modify files manually. Might have to update the Devices Kernel to the stable aosp 4.14 kernel. this is my theory anyway.
[REFERENCE] How to get an Android kernel up to date with linux-stable
Introduction Hello everyone! This will be a thread to assist people with getting their device's Android kernel up to date with the latest linux-stable tag from kernel.org. This process will henceforth be referred to as "upstreaming". This thread...
forum.xda-developers.com
Squida said:
Spoiler: Reply
You sir, are a Legend!
Okay So if I am reading that right.
If you mean DT_OVERLAY=y in defconfig, makes it so I can get the dtbo.img file. I tried adding the dtbo.img to the list of files for extraction and with it on, no dtbo image is made nor found, with it off. still the same thing but unable to extract any dtb or dtbo images from the boot directory. have to have it on.
dtbo.img just does not want to extract for me.
Again, using the clang config and not the mi 9 config used on Micode wiki.
the Micode Wiki settings do not work correctly. you lose out on about a GB of stuff and I know its a DTC issue I have been having due to the Qcom spam.
I have reduced the space though so now you can actually scroll through the whole build process, including the spam.
I will definitely be looking into that solution for channel scratch.
Also if you could please show me the config for the SDClang settings you put it to get it to detect. I tried looking over your git page and failed to understand why you have it in boardconfig.mk and if lineage has other files for the device.
Reason being is because on QAEP it talks about SDclang-3.8 and i have no idea where it is.. Supposed to be in QQ-LLVM-compiler and you copy it to the prebuilts folders, it just does not exist to do that.
the major problem I have is building the device tree when building the kernel. from my understanding, Xiaomi have set it up so DTC creates a Device tree for you. well that is the part that is failing and the whole reason why I can not make a rom.
My guess because you are using lineage sources, everything I am talking about. they already did for you. I am trying to do it all manually. learning purposes.
Click to expand...
Click to collapse
Spoiler: Reply
My guess because you are using lineage sources, everything I am talking about. they already did for you. I am trying to do it all manually. learning purposes.
Click to expand...
Click to collapse
Of course, this is the reason. I don't know exactly how the lineage build scripts generate dtbo.img. There's make bootimage, make vendorimage and maybe make dtboimage too. Probably this command helps you generate it, but I don't think I had success with that: https://forum.xda-developers.com/t/...nel-dtbo-for-redmi-k20.3973787/#post-80354635
The image size differs and the one created by lineage build scripts is the same size as the original one.
I also have console spam when the device tree is generated, I just didnt sweat on it as being an error.
BoardConfig is probably central to the lineage build, but I assume the variables set here are available to kernel's make commands.
For a lineage-less built I used the first two commands here: https://gist.github.com/b100dian/40c8dbe746ff181aff71ee10a75a5f3c (the rest of the things are my attempts to construct back the boot.img with the kernel).
To actually boot the kernel you can gzip it, and append the dtb file to it (
like
Code:
cat Image.gz dtb > Image.gz-dtb
).
, and _then_ reconstruct a deconstructed original boot.img with that (w/o the --dtb parameter if I remember correctly). But I think kernel output already has -dtb concatenated in out/arch/arm64/boot
b100dian said:
Spoiler: Reply
Of course, this is the reason. I don't know exactly how the lineage build scripts generate dtbo.img. There's make bootimage, make vendorimage and maybe make dtboimage too. Probably this command helps you generate it, but I don't think I had success with that: https://forum.xda-developers.com/t/...nel-dtbo-for-redmi-k20.3973787/#post-80354635
The image size differs and the one created by lineage build scripts is the same size as the original one.
I also have console spam when the device tree is generated, I just didnt sweat on it as being an error.
BoardConfig is probably central to the lineage build, but I assume the variables set here are available to kernel's make commands.
For a lineage-less built I used the first two commands here: https://gist.github.com/b100dian/40c8dbe746ff181aff71ee10a75a5f3c (the rest of the things are my attempts to construct back the boot.img with the kernel).
To actually boot the kernel you can gzip it, and append the dtb file to it (
like
Code:
cat Image.gz dtb > Image.gz-dtb
).
, and _then_ reconstruct a deconstructed original boot.img with that (w/o the --dtb parameter if I remember correctly). But I think kernel output already has -dtb concatenated in out/arch/arm64/boot
Click to expand...
Click to collapse
Thanks for the quick reply.
Spoiler: Might be the solution to DTC
Code:
DTC_EXT=~/android/lineage/prebuilts/tools-lineage/linux-x86/dtc/dtc ARCH=arm64 SUBARCH=arm64 CROSS_COMPILE=${PWD}/toolchain/bin/aarch64-linux-android- make O=../tucana-out REAL_CC=${PWD}/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu - vendor/tucana_user_defconfig
DTC_EXT=~/android/lineage/prebuilts/tools-lineage/linux-x86/dtc/dtc ARCH=arm64 SUBARCH=arm64 CROSS_COMPILE=${PWD}/toolchain/bin/aarch64-linux-android- make -j8 O=../tucana-out/ REAL_CC=${PWD}/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- 2>&1 | tee ../kernel.log
Above solution might be the key thing to solving my issues with DTC. i can safely say I never once put the ARCH= and SUBARCH= in DTC_EXT= this may have solved it.
As for the rom building itself, I may have to use AOSP over QAEP, due to the whole SDCLANG issue
Link:
https://developer.qualcomm.com/forum/qdn-forums/software/snapdragon-llvm-compiler-android/68403#comment-18264
Those issues are all related to QAEP, I have less problems with AOSP.
Just don't have a device tree for it. that is why I need DTC working perfectly.
For what it is worth, tried looking for a default device tree that was not based off lineage-os. No luck thus far.
But isn't the device tree what's in arch/arm64/boot/dts/qcom/ in the kernel sources?
There's even a Makefile there where you can see CONFIG_BUILD_ARM64_DT_OVERLAY in use
b100dian said:
But isn't the device tree what's in arch/arm64/boot/dts/qcom/ in the kernel sources?
There's even a Makefile there where you can see CONFIG_BUILD_ARM64_DT_OVERLAY in use
Click to expand...
Click to collapse
Spoiler: Reply
Lmao holy ****, I think I know where I stuffed up. I was navigating extract only at boot. Did not go further then boot directory lol. I'll test now with navigating to qcom. Thanks for the tip.
***UPDATE***
Managed to work out why all these problems are occurring.
Okay so from what I understand, there is 2 building methods.
1. Using terminal and using export commands.
2. build.config file
To test the theory to make sure that indeed there is a double up on builds, I decided to put the whole Micode Mi 9 export guide into the build.config that I have (made a backup of it) after executing it did indeed show errors.
so then I investigate how it is failing and it came to my attention that the export method for the make defconfig and then build is seperate for build.config.
for the kbuild, we issue the environment variables for example, these below.
When I say its Kbuild variables, it could be clang or gcc. Not 100% sure though.
Spoiler: Kbuild Environment Variables
# PRE_DEFCONFIG_CMDS
# Command evaluated before `make defconfig`
#
# POST_DEFCONFIG_CMDS
# Command evaluated after `make defconfig` and before `make`.
#
# POST_KERNEL_BUILD_CMDS
# Command evaluated after `make`.
#
# EXTRA_CMDS
# Command evaluated after building and installing kernel and modules.
#
# DIST_CMDS
# Command evaluated after copying files to DIST_DIR
#
# VENDOR_RAMDISK_CMDS
# When building vendor boot image, VENDOR_RAMDISK_CMDS enables the build
# config file to specify command(s) for further altering the prebuilt vendor
# ramdisk binary. For example, the build config file could add firmware files
# on the vendor ramdisk (lib/firmware) for testing purposes.
Spoiler: extra-info
and instead of having "make" command by itself I put it for example in PRE_DEFCONFIG_CMDS="make <code>*
and it will add in REAL_CC, etc to the defconfig and upon making the build it should inturn do the rest.
Does anyone know the environment variable for kernel build so i can then put in the make command for the build and include REAL_CC=, etc. I managed to get the first make command for the defconfig working.
PRE_DEFCONFIG_CMDS="make O=out/android-4.14 REAL_CC=${PWD}/prebuilts-master/ndk/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- tucana_user_defconfig"
I just now have to get the below command working. not sure what environment variable to put it under.
make -j$(nproc) O=out/android-4.14 REAL_CC=${PWD}/prebuilts-master/ndk/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- 2>&1 | tee kernel.log
Found it!
By searching in build directory then build.sh file.
and *_setup_env.sh* has settings as well that will help.
***********************************
export MAKE_ARGS=$*
Could be useful
CC_LD_ARG
***********************************
echo "========================================================"
echo " Building kernel"
set -x
(cd ${OUT_DIR} && make O=${OUT_DIR} ${CC_LD_ARG} ${MAKE_ARGS})
set +x
***Linux update script***
GitHub - android-linux-stable/script: A script to help with merging linux-stable into your own repository
A script to help with merging linux-stable into your own repository - GitHub - android-linux-stable/script: A script to help with merging linux-stable into your own repository
github.com
Better to use CAF for MSM devices.
[REFERENCE] Merge latest CAF Tag in Kernel
Introduction: Hello folks! In this thread I will be guiding you about how you can merge latest CAF tags in your CAF based kernel. Many people who just started compiling the kernels still don't know how to merge a CAF tag because there isn't any...
forum.xda-developers.com
Upon further research on the topic of MAKE_ARGS=$*
from what I now understand and please keep in mind, I am in no way a programmer or good with Linux overall.
I think the $* symbol is a value itself, so from my understanding. If i use lets say $S=Something I have just made an variable that will be used in MAKE_ARGS=$*
If my theory for this is correct, this is how I add in the make variables for the last build command to start the build.
Should be able to do this.
make -j$(nproc) O=out/android-4.14 REAL_CC=${PWD}/prebuilts-master/ndk/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- 2>&1 | tee kernel.log
Oh and the config is make commands, found that out as well. you have MAKE and SOONG commands and configs. I so prefer Make variables over Soong. looks easier to read for me.
Found a build config on mi code, sources, issues section
SOURCE
Spoiler: build.config-mi11
#!/bin/bash
export OUT=${PWD}/out
export ARCH=arm64
export SUBARCH=arm64
export TARGET_BUILD_VARIANT=userdebug
#export DTC_EXT=dtc
export CROSS_COMPILE=${PWD}/toolchains/aarch64-linux-android-4.9/bin/aarch64-linux-android-
export KERNEL_DEFCONFIG=venus-qgki_defconfig
#set CONFIG_BUILD_ARM64_DT_OVERLAY=y
#set BUILD_CONFIG=build.config.gki.aarch64
O=$OUT REAL_CC=${PWD}/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- ${PWD}/scripts/gki/generate_defconfig.sh $KERNEL_DEFCONFIG
make O=$OUT REAL_CC=${PWD}/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- vendor/$KERNEL_DEFCONFIG
make -j$(nproc) O=$OUT REAL_CC=${PWD}/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- 2>&1 | tee kernel.log
The build config above is indeed for Mi11, we do not need gsi settings or the 3rd command starting with O=$OUT for Mi note 10 pro
By implementing the changes above, outcome below.
Just now have to resolve kernel issues and this should build with no more issues. it has been a massive adventure.
the errors above are due to remnants of old files from merges that were not undone properly. deleted the devices source and recreated it again. problems are now gone will update on outcome.
I still can not seem to get it completely built. Max Image.gz-dtb file size is 28.5MB so far. and the highest I have gotten. still get VMLINUX issues which tells me just maybe Xiaomi themselves have not updated the build scripts to suite the latest CAF changes.
All speculation at the moment.
Had a look at this area
GitHub - MiCode/kernel_build
Contribute to MiCode/kernel_build development by creating an account on GitHub.
github.com
Turns out, build config for CC9 Pro. does not exist hence why all the problems Using AOSP's.
I believe it needs to be full modified in order to work with Tucana, due to Xiaomi not releasing a build script for it. so another solution would be to modify or try and use another device on the list in the link to be able to build it. not even export using the Mi 9 works so I have to dig deeper on getting a config and build script for it.
Spoiler: Current build.config
ARCH=arm64
SUBARCH=arm64
BRANCH=android-4.14
CLANG_TRIPLE=aarch64-linux-gnu-
CROSS_COMPILE=aarch64-linux-android-
TARGET_BUILD_VARIANT=userdebug
DEFCONFIG=tucana_user_defconfig
SKIP_DEFCONFIG=1
PRE_DEFCONFIG_CMDS="make O=out/android-4.14 REAL_CC=${ROOT_DIR}/prebuilts-master/ndk/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin/clang CLANG_TRIPLE=aarch64-linux-gnu- ${DEFCONFIG} && make O=out/android-4.14 menuconfig"
POST_DEFCONFIG_CMDS=""
KERNEL_DIR=.
EXTRA_CMDS=""
HOSTCC=gcc
CC=clang
DTC_EXT=${ROOT_DIR}/prebuilts-master/misc/linux-x86/dtc/dtc
LINUX_GCC_CROSS_COMPILE_PREBUILTS_BIN=prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin
CLANG_PREBUILT_BIN=prebuilts-master/ndk/toolchains/llvm-Snapdragon_LLVM_for_Android_8.0/prebuilt/linux-x86_64/bin
LZ4_PREBUILTS_BIN=prebuilts-master/misc/linux-x86/lz4
DTC_PREBUILTS_BIN=prebuilts-master/misc/linux-x86/dtc
LIBUFDT_PREBUILTS_BIN=prebuilts-master/misc/linux-x86/libufdt
BUILDTOOLS_PREBUILT_BIN=build/build-tools/path/linux-x86
OUT_DIR=out/android-4.14
FILES="
arch/arm64/boot/Image
arch/arm64/boot/Image.gz
arch/arm64/boot/Image.gz-dtb
arch/arm64/boot/dts/qcom/tucana-sdmmagpie-overlay.dtbo
arch/arm64/boot/dts/qcom/sdmmagpie.dtb
vmlinux
System.map
.config
"
EXT_MODULES="
private/msm-tucana-modules/wlan/qcacld-3.0
"
IN_KERNEL_MODULES=1
STOP_SHIP_TRACEPRINTK=1
The config does not build vmlinux, DTC properly, dtbo.img. Still working on it.
I also did try first doing the standalone export way. Just constantly getting ld error --fix-something and to fix it, by using CC=clang and HOSTCC=gcc
but then you resort to having to use the build.config file. for some reason the standalone export method just does not work with cc and hostcc
and that is using the QQ LLVM/clang 8.0 toolchain with AOSP's GCC while having 2 directories. toolchain(GCC) and toolchains(QQ-LLVM/Clang) with QQ-llvm/clang's build directory inside the devices source root directory as well, according to MiCode Wiki under MSM-4.14.
Tried attaching the last make command to the DTC_EXT= and it fails, still unable to find the build kernel argument to change the command manually. Only have POST_KERNEL_BUILD
No luck at all with the export combo.
As for the build.config.
I get a build using AOSP'S GCC and QQ-LLVM/CLANG 8.0
But it is not complete.
I also use the build, prebuilts, prebuilts-master from AOSP's 4.14 Common kernel repo.
the reason for this, is because Xiaomi have not released kernel build files for the device. from what I have found.
Hey, does anyone know how to get the propietary blobs out of the device? I'm kinda done with MIUI and i wanna try to compile some custom rom and maybe a legit twrp.
SanHelios said:
Hey, does anyone know how to get the propietary blobs out of the device? I'm kinda done with MIUI and i wanna try to compile some custom rom and maybe a legit twrp.
Click to expand...
Click to collapse
lol i am looking for the same
check this out
vamsi209 said:
lol i am looking for the same
check this out
Click to expand...
Click to collapse
what shall i check out?
SanHelios said:
what shall i check out?
Click to expand...
Click to collapse
Extracting proprietary blobs from LineageOS zip files | LineageOS Wiki
wiki.lineageos.org
SanHelios said:
Hey, does anyone know how to get the propietary blobs out of the device? I'm kinda done with MIUI and i wanna try to compile some custom rom and maybe a legit twrp.
Click to expand...
Click to collapse
Trying to look for the Chinese tool to flash the unofficial TWRP, once I manage to do that will try help on grabbing those needed proprietary blobs. May need guide on how to pull the blobs, am still a noob
dan079 said:
Trying to look for the Chinese tool to flash the unofficial TWRP, once I manage to do that will try help on grabbing those needed proprietary blobs. May need guide on how to pull the blobs, am still a noob
Click to expand...
Click to collapse
Me too... TWRP is tricky, since it can only be done by this OneInject-function of TWRP, but it's possible. I tried the 'current' unofficial release of TWRP for this, but all i got was a reboot to BL.
vamsi209 said:
Extracting proprietary blobs from LineageOS zip files | LineageOS Wiki
wiki.lineageos.org
Click to expand...
Click to collapse
yshalsager, who created the Firmware-script, told me the same.. here is his answer.
"
Hi,
Thanks for your words, glad to hear my work helps.
You can use LineageOS extract files script that will generate vendor tree for you. It is available in any device tree but you should use one of your device so it reads from its proprietary-files.txt or something.
"
Maybe LOS is closer than we think.
SanHelios said:
yshalsager, who created the Firmware-script, told me the same.. here is his answer.
"
Hi,
Thanks for your words, glad to hear my work helps.
You can use LineageOS extract files script that will generate vendor tree for you. It is available in any device tree but you should use one of your device so it reads from its proprietary-files.txt or something.
"
Maybe LOS is closer than we think.
Click to expand...
Click to collapse
niceee, so for mix 4, i extracted the twrp trees using this,
[SCRIPT] TWRP device tree generator
Create a TWRP-compatible device tree only from an Android recovery image (or a boot image if the device uses non-dynamic partitions A/B) of your device's stock ROM. It has been confirmed that this script supports images built starting from...
forum.xda-developers.com
setup the local repo for building twrp trees, using these
1. Installing the tools
A Python library/script to automatically generate TWRP-compatible device tree from a boot/recovery image - twrpdtgen/twrpdtgen
github-wiki-see.page
tried building but the device doesn't lunch after following the steps shown above,
Code:
http://www.hastebin.com/jonexiyowu.md
and one of the dev https://github.com/imjyotiraditya , helped me build an aospa rom for g8x,
now he is on to building twrp for our device odin.
if anyone has twrp trees, we can try geting aospa build ready for our device
vamsi209 said:
niceee, so for mix 4, i extracted the twrp trees using this,
[SCRIPT] TWRP device tree generator
Create a TWRP-compatible device tree only from an Android recovery image (or a boot image if the device uses non-dynamic partitions A/B) of your device's stock ROM. It has been confirmed that this script supports images built starting from...
forum.xda-developers.com
setup the local repo for building twrp trees, using these
1. Installing the tools
A Python library/script to automatically generate TWRP-compatible device tree from a boot/recovery image - twrpdtgen/twrpdtgen
github-wiki-see.page
tried building but the device doesn't lunch after following the steps shown above,
Code:
http://www.hastebin.com/jonexiyowu.md
and one of the dev https://github.com/imjyotiraditya , helped me build an aospa rom for g8x,
now he is on to building twrp for our device odin.
if anyone has twrp trees, we can try geting aospa build ready for our device
Click to expand...
Click to collapse
hey, i'm trying it right now... was able to manage a device tree from the latest weekly of the EU-rom. Repo is syncing right now for the aosp-twrp-11 repository.
U used the same script for the recovery trees?
vamsi209 said:
U used the same script for the recovery trees?
Click to expand...
Click to collapse
yes, i extracted it from the boot.img.
Update... build/envsetup.sh error, anyone any suggestions?
Update managed to get envsetup.sh to work, got following error messages
source build/envsetup.sh
including device/xiaomi/odin/vendorsetup.sh
COMMON_LUNCH_CHOICES: Befehl nicht gefunden.
COMMON_LUNCH_CHOICES: Befehl nicht gefunden.
lunch twrp_odin-eng
In file included from build/make/core/config.mk:291:
In file included from build/make/core/envsetup.mk:266:
build/make/core/product_config.mk:155: error: Can not locate config makefile for product "twrp_odin".
23:22:04 dumpvars failed with: exit status 1
WARNING: Trying to fetch a device that's already there
Traceback (most recent call last):
File "/home/dave/AOSP-Recovery/vendor/twrp/build/tools/roomservice.py", line 431, in <module>
fetch_device(device)
File "/home/dave/AOSP-Recovery/vendor/twrp/build/tools/roomservice.py", line 399, in fetch_device
git_data = search_gerrit_for_device(device)
File "/home/dave/AOSP-Recovery/vendor/twrp/build/tools/roomservice.py", line 86, in search_gerrit_for_device
device_data = check_repo_exists(git_data, device)
File "/home/dave/AOSP-Recovery/vendor/twrp/build/tools/roomservice.py", line 62, in check_repo_exists
raise Exception("{device} not found,"
Exception: odin not found,exiting roomservice
In file included from build/make/core/config.mk:291:
In file included from build/make/core/envsetup.mk:266:
build/make/core/product_config.mk:155: error: Can not locate config makefile for product "twrp_odin".
23:22:05 dumpvars failed with: exit status 1
** Don't have a product spec for: 'twrp_odin'
** Do you have the right repo manifest?
Anyone a good guess?
Did you had any luck, or any chance I can help?
Puksom said:
Did you had any luck, or any chance I can help?
Click to expand...
Click to collapse
well, i made some progress, but i failed again. I posted a thread in the official twrp forum. Maybe you might want to take a look at it.. thx.
Post in thread '[DEV]How to compile TWRP touch recovery' https://forum.xda-developers.com/t/dev-how-to-compile-twrp-touch-recovery.1943625/post-85686505
Puksom said:
Did you had any luck, or any chance I can help?
Click to expand...
Click to collapse
I'm a total beginner, so i might be wrong. But as far as i can tell, the makefiles of the extracted device tree need to be update or even completely rebuild.
Hi, it seems like I was able to execute the script successfully.
It didn't work on Windows because it got stuck on the execution of unpackimg.bat.
I ran it on Linux and it worked (after chmod 777 of boot.img). This is the command I ran:
python3 -m twrpdtgen -o ./odin ./boot.img
output:
Code:
TWRP device tree generator
Version 1.3.0
[INFO] Cloning AIK...
Done! You can find the device tree in odin/xiaomi/odin
I took boot.img from the latest MIUI 12.5.7.0 China Stable
Now I have what it seems to be the device tree (odin.zip) but I don't know what to do it it or what it is.
radoinc said:
Hi, it seems like I was able to execute the script successfully.
It didn't work on Windows because it got stuck on the execution of unpackimg.bat.
I ran it on Linux and it worked (after chmod 777 of boot.img). This is the command I ran:
python3 -m twrpdtgen -o ./odin ./boot.img
output:
Code:
TWRP device tree generator
Version 1.3.0
[INFO] Cloning AIK...
Done! You can find the device tree in odin/xiaomi/odin
I took boot.img from the latest MIUI 12.5.7.0 China Stable
Now I have what it seems to be the device tree (odin.zip) but I don't know what to do it it or what it is.
Click to expand...
Click to collapse
This is great, so we know this script works and does what it is supposed to do...
I checked the makefiles to see if there are any differences, but there are none. So it suggests that it doesn't matter, from which version you get the files from. I love, that the users of the Mi Mix 4 are more open so compiling than the community of the MI 11 Ultra is..
SanHelios said:
This is great, so we know this script works and does what it is supposed to do...
I checked the makefiles to see if there are any differences, but there are none. So it suggests that it doesn't matter, from which version you get the files from. I love, that the users of the Mi Mix 4 are more open so compiling than the community of the MI 11 Ultra is..
Click to expand...
Click to collapse
I'm trying to replicate your steps but I get the same "Can not locate config makefile for product "twrp_odin"." as you did above. In the other topic I see you managed to get past that step.
I see you get some output after executing envsetup.sh and it seems like this is related to the device tree.
Can you please share what you did with the device tree before attempting to compile twrp? I'd like to try myself but I can't find clear instructions.
radoinc said:
I'm trying to replicate your steps but I get the same "Can not locate config makefile for product "twrp_odin"." as you did above. In the other topic I see you managed to get past that step.
I see you get some output after executing envsetup.sh and it seems like this is related to the device tree.
Can you please share what you did with the device tree before attempting to compile twrp? I'd like to try myself but I can't find clear instructions.
Click to expand...
Click to collapse
Acutally i left it completely unchanged, the only thing i did was to change the repository. I deleted the aosp-repository and took the omni-twrp-repository
(https://github.com/minimal-manifest-twrp/platform_manifest_twrp_omni)
mkdir twrp
cd twrp
repo init -u git://github.com/minimal-manifest-twrp/platform_manifest_twrp_omni.git -b twrp-10.0
repo sync
after syncing was complete, i followed these instructions
4. Build TWRP from source
A Python library/script to automatically generate TWRP-compatible device tree from a boot/recovery image - twrpdtgen/twrpdtgen
github-wiki-see.page
SanHelios said:
Acutally i left it completely unchanged, the only thing i did was to change the repository. I deleted the aosp-repository and took the omni-twrp-repository
(https://github.com/minimal-manifest-twrp/platform_manifest_twrp_omni)
mkdir twrp
cd twrp
repo init -u git://github.com/minimal-manifest-twrp/platform_manifest_twrp_omni.git -b twrp-10.0
repo sync
after syncing was complete, i followed these instructions
4. Build TWRP from source
A Python library/script to automatically generate TWRP-compatible device tree from a boot/recovery image - twrpdtgen/twrpdtgen
github-wiki-see.page
Click to expand...
Click to collapse
Thanks! Now I managed to get the same result as you. The error seems to be raised by mkbootimg.py:
Python:
def write_header(args):
BOOT_IMAGE_HEADER_V1_SIZE = 1648
BOOT_IMAGE_HEADER_V2_SIZE = 1660
BOOT_MAGIC = 'ANDROID!'.encode()
if (args.header_version > 2):
raise ValueError('Boot header version %d not supported' % args.header_version)
To me it seems like this Omni repo includes old version of mkbootimg.py, because in the google repos I can see that the current version of this function looks like this:
Python:
def write_header(args):
if args.header_version > 4:
raise ValueError(
f'Boot header version {args.header_version} not supported')
if args.header_version in {3, 4}:
return write_header_v3_and_above(args)
It seems like Boot header version 3 was introduced with Android 11: https://source.android.com/devices/bootloader/boot-image-header
I think we can't do much with the Omnia repos until they get updated with current mkbootimg.
radoinc said:
Thanks! Now I managed to get the same result as you. The error seems to be raised by mkbootimg.py:
Python:
def write_header(args):
BOOT_IMAGE_HEADER_V1_SIZE = 1648
BOOT_IMAGE_HEADER_V2_SIZE = 1660
BOOT_MAGIC = 'ANDROID!'.encode()
if (args.header_version > 2):
raise ValueError('Boot header version %d not supported' % args.header_version)
To me it seems like this Omni repo includes old version of mkbootimg.py, because in the google repos I can see that the current version of this function looks like this:
Python:
def write_header(args):
if args.header_version > 4:
raise ValueError(
f'Boot header version {args.header_version} not supported')
if args.header_version in {3, 4}:
return write_header_v3_and_above(args)
It seems like Boot header version 3 was introduced with Android 11: https://source.android.com/devices/bootloader/boot-image-header
I think we can't do much with the Omnia repos until they get updated with current mkbootimg.
Click to expand...
Click to collapse
My question is, do the makefiles from the devicetree need to be adjusted or completely rebuild to android 11 parameters? I.e. rhe command "add_lunch_combo" is obsolete and "COMMAND_LUNCH_CHOICES" took its place..
Sorry in advance for this nooby questions...