Introduction
Hello all, I am bringing you this thread as a jumping off point to compiling kernels and working with upstream Linux. I will include a guide, some links, and some terms that will help you get started with modifying kernel source. This is also a good reference point for existing developers as I have consolidated all upstream patches into one repo. Let's get down into it!
What in the world is upstream Linux?
When an OEM sets up a device, they will pick a stable longterm branch from the Linux kernel to base their modifications around (drivers and such). In the case of Shamu, they picked 3.10.40. Currently, the Linux kernel's 3.10 is updated to 3.10.105, as you can see on kernel.org. This means that Google is "missing" versions 3.10.41 to 3.10.105. Now, why does this matter? Well, the way that the Linux kernel runs its stable branches, the only things that get merged into there are bug fixes and security updates. That's it, there are no wonky features or unstable patches. The only way you get a patch into a stable branch is by having it be in the mainline branch first. Some developers have an aversion to adding upstream because they claim it is excessive and not necessary and they are partially right since not all the patches that come in are relevant to our architecture (arm). However, upstream Linux is not unstable and by adding each version one at a time, you can verify this. Being up to date is good since you keep yourself protected from bugs and security issues that crop up. Google has been better about doing this lately with their monthly security updates but it never hurts to take matters into your own hands.
I suggest watching one of these talks given by Greg Kroah-Hartman, it is really interesting to see how the process goes:
https://www.youtube.com/watch?v=SPY0LyTU53w | https://www.youtube.com/watch?v=L2SED6sewRw
What did you do?
All I did was fetch the latest kernel.org patches from here and cherry pick them on top of the latest kernel source from Google.
I merge these patches by cherry-picking each version individually (3.10.40 to 3.10.41, 3.10.41 to 3.10.42, etc), that way I can verify that the kernel compiles fine and that there are no merge conflicts. Google will sometimes pick certain commits from upstream that are of a higher importance than others which can result in conflicts if you try to pick it again. Additionally, upstream might fix a bug one way and Google has done it another (which is not really good, Google should be pushing their fixes back to upstream so everything stays in sync).
What do I do with this?
I have created two repos below: one with the latest N security update branch with the latest upstream patches picked up on top of it (the shamu-upstream branch) and another one with a plain AnyKernel source for you to modify (shamu-stock is the most basic, shamu-decrypt contains an fstab file that will disable both forced encryption and dm-verity). You are free to fork these or base other branches on them, that's the whole point of this post. I have verified that all the patches contain no major detectable issues. I would like some credit if you do use it but it's not required since the kernel is licensed under GPL Another reason I offer this is I have seen a lot of developers picking in upstream in patch sets, so you get one single commit for an upstream version. This is detrimental as you start to add your own patches as you cannot fully tell what was modified and for what reason without the individual commits. It might look cleaner but you don't get full history which hurts you in the long run.
Links
Kernel source: https://github.com/nathanchance/shamu/tree/upstream-7.1.1
AnyKernel source: https://github.com/nathanchance/AnyKernel2/tree/shamu-stock-decrypt
Toolchain source: https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/arm/arm-eabi-4.8/
How to compile
This will be a quick, step by step guide on how to compile this kernel from source. By using this process, you can start to make modifications to the kernel source and make a flashable zip.
Clone the kernel source, the AnyKernel source, and toolchain source
Code:
cd ~
mkdir Kernel && cd Kernel
git clone https://github.com/nathanchance/shamu.git source
git clone https://github.com/nathanchance/AnyKernel2.git anykernel
git clone https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/arm/arm-eabi-4.8 AOSP-4.8
Explanation:
First command: make sure you are in your home directory (or whatever directory you want to hold the kernel folder we are about to make.
Second command: make a Kernel folder and move into it
Third command: clone the kernel source from my repo into a folder named source
Fourth command: clone the AnyKernel source from my repo into a folder named anykernel
Fifth command: clone the Google 4.8 toolchain into a folder named AOSP-4.8
AnyKernel is the name of the zip we are going to make which allows the kernel to be flashed over any ROM.
A toolchain is a set of compiler tools that allow us to compile the kernel on any computer architecture.
Make sure you are on the correct branches
Code:
cd source && git checkout upstream-7.1.1
cd ../anykernel && git checkout shamu-stock-decrypt
Explanation:
First command: move into the source directory and checkout the branch "upstream-7.1.1", which is the necessary branch for this device.
Second command: move into the AnyKernel directory and checkout the branch "shamu-decrypt". If you don't want to disable forced encryption, use the "shamu-stock" branch.
Tell the compiler what you are are compiling
Code:
export CROSS_COMPILE=${HOME}/Kernel/AOSP-4.8/bin/arm-eabi-
export ARCH=arm && export SUBARCH=arm
make clean && make mrproper
make shamu_defconfig
Explanation:
First command: point the compiler to the location of your toolchain. If you have done anything different with the folder locations, you will need to modify the "${HOME}/Kernel/AOSP-4.8" part.
Second command: tell the compiler which architecture we are compiling for. In this case, our device is an arm device.
Third command: clean out any compiled files and remove our previous defconfig.
Fourth command: tell the compiler which options we want in the kernel using the shamu_defconfig.
A defconfig is a file that will tell the compiler which features we want in the kernel. No computer ever uses all of the options in the kernel since there are different drivers for various devices.
Make the kernel!
Code:
make -j$(nproc --all)
Explanation:
make tells the compiler to make the kernel (duh) and the -j$(nproc --all) tells the compiler to use the maximum number of cores your computer has available
Make the AnyKernel zip
Code:
cp -v arch/arm/boot/zImage-dtb ../anykernel
cd ../anykernel
zip -r9 stock-upstream.zip * -x README stock-upstream.zip
Explanation:
First command: copies the completed kernel (zImage-dtb) into the AnyKernel folder
Second command: moves us into the AnyKernel folder
Third command: makes the zip file (named stock-upstream.zip) in the AnyKernel folder.
If you ever want to do this again, run these commands and go straight to step 3:
Code:
cd source && git clean -fxd && git pull
cd anykernel && git clean -fxd && git pull
NOTE: This thread is aimed to be a breeding ground for kernel development, a place to jump off if you will. I am happy to answer how to questions about building kernels or flashing the one I have provided but this is not supposed to be a general Q&A thread. Please use another thread or create your own in Q&A if you need assistance.
Nice write up.
One suggestion though. Instead of using the current version as the branch name, you may want to just name them for the device, like "Shamu-Upstream" then tag the versions once you've gotten to the version commit. This will prevent you from having to update the thread for this command anytime that you pull in more upstream changes. It also makes it alot cleaner/easier if someone for whatever reason wanted to checkout a specific patch version as they could just checkout the tag rather than resetting back to the commit.
For lack of a better explanation, tags are sort of like sub-branches within a branch.
Very new to kernel building and AOSP in general. Followed the guide but I'm getting stuck when make mrproper is ran. Build stops and i get the error : unknown target 'mrproper' then the build fails. Any help?
The Flash said:
Another reason I offer this is I have seen a lot of developers picking in upstream in patch sets, so you get one single commit for an upstream version. This is detrimental as you start to add your own patches as you cannot fully tell what was modified and for what reason without the individual commits. It might look cleaner but you don't get full history which hurts you in the long run.
Click to expand...
Click to collapse
That's true. Some devs are lazy to commit each patch and bring up from 3.10.40 to 3.10.103 which comprises nearly 3000 patches.
neobuddy89 said:
That's true. Some devs are lazy to commit each patch and bring up from 3.10.40 to 3.10.103 which comprises nearly 3000 patches.
Click to expand...
Click to collapse
Exactly lol. It's also annoying for people looking at your source because if you had to fix anything, it is going to be lumped in with that enormous patch.
Root Wizard said:
Very new to kernel building and AOSP in general. Followed the guide but I'm getting stuck when make mrproper is ran. Build stops and i get the error : unknown target 'mrproper' then the build fails. Any help?
Click to expand...
Click to collapse
can I see a full terminal window/log showing you entering all the commands?
The Flash said:
can I see a full terminal window/log showing you entering all the commands?
Click to expand...
Click to collapse
Just logged off for the night. Will give a full report back tomorrow with pictures. Basically all the commands go good up to 'make mrproper' . 'make clean' runs fine but the 'make mrproper' goes through incuding all the aosp files for build, then ninja enters build directory, then the error 'unknown target 'mrproper' ' comes up, and the build fails.
Root Wizard said:
Just logged off for the night. Will give a full report back tomorrow with pictures. Basically all the commands go good up to 'make mrproper' . 'make clean' runs fine but the 'make mrproper' goes through incuding all the aosp files for build, then ninja enters build directory, then the error 'unknown target 'mrproper' ' comes up, and the build fails.
Click to expand...
Click to collapse
Oh you're building this into a ROM? Alright, that might make things different but yeah show me pictures tomorrow and I'll try to get you up and running.
The Flash said:
Oh you're building this into a ROM? Alright, that might make things different but yeah show me pictures tomorrow and I'll try to get you up and running.
Click to expand...
Click to collapse
Ah I guess I should have metioned that I was building an aosp rom. Thanks for your quick assistance. Will get back tomorrow.
If you're building into a rom, you just need to change the existing kernel source to this one in the xml for this device.
Yoinx said:
If you're building into a rom, you just need to change the existing kernel source to this one in the xml for this device.
Click to expand...
Click to collapse
Can you explain how to do this? First time building with aosp, not very familiar which xml file or what to change to apply the upstream patch.
Root Wizard said:
Can you explain how to do this? First time building with aosp, not very familiar which xml file or what to change to apply the upstream patch.
Click to expand...
Click to collapse
For shamu in aosp, the kernel is pulled with the main aosp source by that manifest. https://github.com/android/platform_manifest/blob/master/default.xml#L65
You world need to substitute this repo with the new one, and delete your existing kernel source. Then next time you sync the repo tool will pull it down.
On other roms it gets pulled in during the breakfast/brunch process. I can't remember exactly where that comes from, but I think that is in the vendor directory for the device, maybe.
How to modify rom manifests and such though is a little outside of this thread's topic. If you need more info I'd recommend opening a thread in the Q&A forum unless this is something that The flash wants to cover in this thread.
There are a few other caveats with doing it in the rom as well. The defconfig name would need to match what the rom build expects and you would need to go in and checkout the correct branch.
Yoinx said:
For shamu in aosp, the kernel is pulled with the main aosp source by that manifest. https://github.com/android/platform_manifest/blob/master/default.xml#L65
You world need to substitute this repo with the new one, and delete your existing kernel source. Then next time you sync the repo tool will pull it down.
On other roms it gets pulled in during the breakfast/brunch process. I can't remember exactly where that comes from, but I think that is in the vendor directory for the device, maybe.
How to modify rom manifests and such though is a little outside of this thread's topic. If you need more info I'd recommend opening a thread in the Q&A forum unless this is something that The flash wants to cover in this thread.
There are a few other caveats with doing it in the rom as well. The defconfig name would need to match what the rom build expects and you would need to go in and checkout the correct branch.
Click to expand...
Click to collapse
Ok, thanks for your help.
For those of you who have forked my kernel, you will need to revert this patch: https://github.com/nathanchance/and...mmit/0b17006c16bba286ecc9d7bacd75a3902a26835e
It causes issues with Wi-Fi and Chrome.
The Flash said:
For those of you who have forked my kernel, you will need to revert this patch: https://github.com/nathanchance/and...mmit/0b17006c16bba286ecc9d7bacd75a3902a26835e
It causes issues with Wi-Fi and Chrome.
Click to expand...
Click to collapse
It does? I have been using Chrome Beta (on wifi) since last night fine with that patch. Maybe its just a matter of time before things start failing. Thanks anyway, will keep a heads up.
Root Wizard said:
It does? I have been using Chrome Beta (on wifi) since last night fine with that patch. Maybe its just a matter of time before things start failing. Thanks anyway, will keep a heads up.
Click to expand...
Click to collapse
If you toggle Wi-Fi, start flipping through pages in Chrome, and toggle Wi-Fi again, it will reboot. Oddest issue I have ever dealt with.
Sent from my Nexus 6P using XDA Labs
The Flash said:
If you toggle Wi-Fi, start flipping through pages in Chrome, and toggle Wi-Fi again, it will reboot. Oddest issue I have ever dealt with.
Click to expand...
Click to collapse
Interesting, will look into this. I also have been getting reboots from searching forums in xda labs app. Logcat says a ton of "PowerHAL_touch_boost failed to send: Connection refused" before reboots. Is this what is happening to you?
Root Wizard said:
Interesting, will look into this. I also have been getting reboots from searching forums in xda labs app. Logcat says a ton of "PowerHAL_touch_boost failed to send: Connection refused" before reboots. Is this what is happening to you?
Click to expand...
Click to collapse
No I haven't had that happen but that error shouldn't cause reboots.
The Flash said:
No I haven't had that happen but that error shouldn't cause reboots.
Click to expand...
Click to collapse
Huh, odd
Github and the kernel in the OP are updated to 3.10.104.
Related
I'm not sure if this is the place to ask this, but I've been playing with building AOSP over the past week. I've got everything synced, ccache, and have built it 4 times. But I have a few questions:
My process: First I "repo init -u <insert proper google url> -b android-4.2.1_r1.2" for the latest 4.2 branch as opposed to just "repo init -u <insert proper google url>" for the master branch, then I do a "repo sync", then I initiate the build "make clean -j9 otapackage".
Questions:
1. By doing repo init with the 4.2.1_r1.2 branch, when I build it, does that ensure that I am building from that branch? So you have to repo init for whatever branch you want to build and the make knows which branch to compile from?
2. If I repo init many branches, and sync those branches to my computer, I can do a make on them later without syncing, as long as I did the init command just before the make command? Of course, I may no longer be in sync with the main repository, but I'll still be building the code as it was previousely? I.e. there is no need to sync before each build as long as you are happy with the code currently on your machine?
3. How often do the branches change? It seems each day I sync and get new projects, etc.
4. Should I do "make -j1" instead of "make -j9"? The latter does the build in under an hour and the former make take several hours. Is it really likely that using multiple thread will cause errors? Could I check that by just compiling the code with "-j9" a few times in a row without a repo sync in order to compare the builds to see if they have the same md5 or something?
I'm still trying to figure out how to get my build to incorporate certain proprietary binaries that are not included in the AOSP. I know I can get these binaries from my phone, or from factory images, or elsewhere (I'm building for my Verizon galaxy nexus - toro). Can I just drop them into the flashable zip? Or is it better to incorporate them into your local code with modifications to the make files that will automate compiling them into the build and otapackage? I have been doing some research on that and think I am almost to the point where I know which make files to modify. Any clues or hints are greatly appreciated.
Also, is there a way to make the otapackage odexed? Or does the phone automatically make the odex files as you run all the apps? <-that's the impression I get after doing some reading.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Introduction
Hello everyone, this is my personal kernel for the Nexus 6P. While I am not a super star kernel developer, I do know my way around it. My philosophy on kernels is pretty simple:
Follow upstream Linux always
Stable kernel rules work best
Security and stability are paramount
Features will be added only as needed
Hopefully you enjoy
Thread Rules
I refuse to acknowledge/respond to bug reports that do not include clear steps to reproduce, a logcat, and a console ramoops (if random rebooting) or a dmesg (any other issues). Follow this guide to do this. More on bug reporting policy here.
Keep things civil, I do NOT tolerate flaming of any kind.
OT is allowed.
Features
Here are the major features of the kernel (feel free to explore Github)
Updated to the latest security patch on the latest Android version (August on Oreo)
Updated to the latest Linux upstream version (3.10.107)
Several CVE fixes
Upstream patches from CAF and Linux
Touchboost force disabled
High brightness mode
KCAL color control
Adreno Boost
Wake gestures (sweep2wake, doubletap2wake, sweep2sleep, and camera launch gesture)
Faux Sound Control with high performance audio
ZRAM from 3.18
Maple I/O scheduler
Wakelock controls
TCP algorithms: bic, cubic, highspeed, htcp, hybia, illinois, lp, reno, scalable, vegas, veno, westwood (default)
F2FS support (latest from here)
Compiled with a Linaro 7.2.1 toolchain built with crosstool-NG (located here).
Makefile optimizations (courtesy of @adinkwok)
Forced encryption and dm-verity disabled
Hides bootloader unlock from SafetyNet (can still fail through other means)
Flashed with AnyKernel2 (thanks to @osm0sis)
Downloads
8.0.0 (v3.x): https://github.com/nathanchance/angler/releases
Legacy downloads:
7.1.2 (v1.x-v2.x): https://github.com/nathanchance/angler/releases
7.1.1 (4.x): Main
7.0 (3.x): Main
6.0.1 (1.x): Main
Do not mirror my kernels!
Installation
If you are on stock:
If you are on the stock kernel: flash the kernel zip and reboot
If you are on a custom kernel: flash the stock boot image matching your build number, flash the kernel zip, and reboot
If you are on a custom ROM:
If you are on the kernel that came with your ROM: flash the kernel zip and reboot
If you are on a custom kernel: dirty flash your ROM zip, flash the kernel zip, and reboot
Upgrading from one Flash version to the other:
Just flash the kernel zip and reboot
Changelog
Latest post
Kernel: https://github.com/nathanchance/angler/commits/8.0.0-flash
AnyKernel: https://github.com/nathanchance/AnyKernel2-archive/commits/angler-flash-public-8.0.0
Frequently asked questions
1.1 Q: I am having an issue, can you help me?
1.1 A: I am not an amazing kernel developer but I will certainly try my best. First, start by reading the release posts, as I may make note of a particular issue. Also search the thread to see if I have discussed it before. If you see nothing, please provide me with a dmesg/last_kmsg and logcat, your ROM/kernel version, and a detailed description of the issue, following this guide. If I can't reproduce reliably though, I'm not going to fix the issue.
1.2 Q: Can you add xyz feature?
1.2 A: If you link me the commits or tell me what kernel you had the feature in, I can take a look at them. I will not go digging super hard for commits and I will ignore requests that don't include this information. I won't promise that I will merge every request but I will always take it into consideration.
1.3 Q: How do I control the kernel?
1.3 A: I'd recommend using either @flar2's EX Kernel Manager or @Yoinx's Kernel Adiutor-Mod; both are extremely solid and have given me no issues.
1.4 Q: What ROM are you on? What are your settings?
1.4 A: Currently, I am on Dirty Unicorns and my settings are available here
1.5 Q: OMFG MY BATTERY SUXXXX PLS FIX!!!
1.5 A: The kernel doesn't drain any battery, use an app like BetterBatteryStats to figure out what app is screwing with your phone.
Thanks to:
@neobuddy89 for always providing great fixes and being a shining example of a great kernel developer
The various other developers for their patches (@ak, @Buckmarble, @flar2, @franciscofranco, @frap129, @imoseyon, and @Yank555 to name a few)
@osm0sis for AnyKernel2
@DespairFactor for helping me with some kernel compilation issues
@kwoktopus for giving me good advice on Makefile flags and other patches
@kantjer for being a beta tester along the way!
@malybru for my nifty new signature!
@phantom146 for the thread banner and default profile GlassCannon
All of you guys for viewing the thread
A note about donations
Quite a few people have asked to donate to me in the past and I have turned them down. I am not in this for the money, this is my hobby, something I truly enjoy. If you truly want to donate to something (it is not expected in the slightest), I recommend an entity like the Open Source Initiative, the Free Software Foundation, XDA, or any one of the people I have thanked in the OP. Additionally, you are free to copy any and all of my work; the only thing I request is that you not ask for donations as well (though I can't really enforce this lol). Thank you.
Kernel source links
8.0.0: https://github.com/nathanchance/angler/tree/8.0.0-flash
7.1.2: https://github.com/nathanchance/angler/tree/7.1.2-flash
7.1.1: https://github.com/nathanchance/angler/tree/7.1.1-flash
7.0: https://github.com/nathanchance/angler-old/tree/release-7.0
6.0.1: https://github.com/nathanchance/angler-old/tree/release-6.0.1
DISCLAIMER: I openly welcome off topic in my threads. Mods, please do not clean off topic discussion and users, do not complain about off topic discussion here. I may direct you to another thread if it is so suited but I won't have the discussion cleaned. Thank you.
XDA:DevDB Information
Flash Kernel, Kernel for the Huawei Nexus 6P
Contributors
nathanchance
Source Code: https://github.com/nathanchance/angler
Kernel Special Features:
Version Information
Status: Stable
Current Stable Version: v3.1
Stable Release Date: 2017-09-17
Created 2016-08-11
Last Updated 2019-07-02
How to compile the kernel
BUILD GUIDE: https://github.com/nathanchance/Android-Tools/blob/master/Guides/Building_Flash.txt
BUILD SCRIPT: https://github.com/nathanchance/Android-Tools/blob/master/Scripts/build-flash.sh
FORMER GUIDE:
1. Configure your environment
In order to compile this kernel, I'd recommend using Linux (OS X should be acceptable but this guide will be tailored for Linux). If you can't dual bot, use Virtual Box to create a virtual machine. To get the environment configured, either use Google's own guide here or my guide (only section 1) here.
2. Grab the source
Grab the kernel source
Code:
mkdir ~/Android
cd ~/Android
git clone https://github.com/Flash-Kernel/angler.git Flash
Explanation:
First command makes a directory named Android in your home folder (mkdir = make directory). You can change the location if you want but keep it consistent throughout the guide.
Second command moves into that directory (change directory)
Third command pulls the source from my Github into a directory named Ninja (git clone <url> <folder_to_clone_in_to> is the format of the command).
Grab the toolchain you want to use to compile the kernel
Code:
cd ~/Android
git clone <kernel_url>
Explanation:
In order to compile the kernel, we need to grab a toolchain (bunch of compilation tools). I currently use DF-LINARO4.9 (third to last option in this list) but I have used all of these in the past. Below are the list of possible URLs to clone; copy and paste it after the words git clone above:
AOSP: https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9
Uber 4.9: https://bitbucket.org/DespairFactor/aarch64-linux-android-4.9-kernel.git
Uber 5.4: https://bitbucket.org/DespairFactor/aarch64-linux-android-5.x-kernel.git
Uber 6.1: https://bitbucket.org/DespairFactor/aarch64-linux-android-6.x-kernel.git
Uber 7.0: https://bitbucket.org/DespairFactor/aarch64-linux-android-7.0-kernel.git
Linaro 4.9: https://android-git.linaro.org/git/.../aarch64/aarch64-linux-android-4.9-linaro.git
Linaro 5.4: https://android-git.linaro.org/git/.../aarch64/aarch64-linux-android-5.4-linaro.git
Linaro 6.1: https://android-git.linaro.org/git/.../aarch64/aarch64-linux-android-6.1-linaro.git
DF-Linaro 4.9: https://bitbucket.org/DespairFactor/aarch64-linux-android-4.9-kernel-linaro.git
DF-Linaro 5.4: https://bitbucket.org/DespairFactor/aarch64-linux-android-5.x-kernel-linaro.git
DF-Linaro 6.1: https://bitbucket.org/DespairFactor/aarch64-linux-android-6.x-kernel-linaro.git
3. Setup the build
Make sure everything looks right
After following the above steps, type:
Code:
ls
You should see something like this:
Code:
aarch64-linux-android-4.9-kernel-linaro Flash
The toolchain directory make be different depending on what you cloned above. If it looks different, delete the Android folder and do step 2 again.
Select the correct kernel branch
Code:
cd ~/Android/Flash
git checkout release
Explanation
First command will move you into the Flash source directory we cloned above.
Second command will checkout the branch you want to compile for (in this case, release)
4. Make the kernel
Tell the makefile what toolchain and architecture you are building for
Figure out the full path of your toolchain folder
Assuming you have done this whole process correctly, you can move into your toolchain directory and type:
Code:
cd ~/Android/<toolchain_folder>
pwd
Example:
Code:
cd ~/Android/aarch64-linux-android-4.9-kernel-linaro
pwd
You should get something like this:
Code:
/home/nathan/Android/aarch64-linux-android-4.9-kernel-linaro
Tell the compiler where that toolchain is
Copy and paste that file location above into this command:
Code:
export CROSS_COMPILE=<toolchain_dir>/bin/aarch64-linux-android-
Example:
Code:
export CROSS_COMPILE=/home/nathan/Android/aarch64-linux-android-4.9-kernel-linaro/bin/aarch64-linux-android-
Set the architecture
Angler is a 64-bit device so we need arm64
Code:
export ARCH=arm64
export SUBARCH=arm64
Clean up from a previous compilation and update the source
Code:
cd ~/Android/Flash
git reset --hard origin/n
git clean -f -d -x > /dev/null 2>&1
make clean && make mrproper
git pull
Example:
Code:
cd ~/Android/Flash
git reset --hard origin/n
git clean -f -d -x > /dev/null 2>&1
make clean && make mrproper
git pull
Explanation:
The first command moves you into the Flash source directory.
The second command will undo any local commits you have done and align your source with the most recent commit from me.
The third command will clean any untracked or compiled files (this is send to /dev/null so you do not see it happen).
The fourth and fifth command (strung together with &&) will remove you
The sixth command will pull any new kernel changes from me into the source.
Set up the defconfig
A defconfig tells the compiler which features you want in the kernel. This is already set up by me. To set up the defconfig, type the following:
Code:
cd ~/Android/Ninja
make flash_defconfig
Name the kernel
If you notice in About Phone > Kernel Version, it shows 3.10.103-FLASH-v3.10 (or something similar). You can customize this by opening the Makefile in the source directory and editing the EXTRAVERSION variable (this will be undone next time you compile).
Make the kernel!
Code:
make -j#
Example:
Code:
make -j4
Explanation:
The # after the "j" is how many simultaneous threads are run. Traditionally, this is the number of cores you have plus one. So a quad-core CPU would be -j5.
5. Zip it up and flash!
Copy the zImage to the AnyKernel directory
Code:
cd ~/Android/Flash/arch/arm64/boot
cp -v Image.gz-dtb ~/Android/Flash/anykernel/zImage
Explanation:
We need to copy the newly compiled kernel (which resides in the arch/arm64/boot folder) to the AnyKernel directory, while changing its name to zImage.
cp = copy
-v = verbose (prints the copying)
Image.gz-dtb = file to be copied
~/Android/Flash/anykernel/zImage = destination and name
Zip it up!
Code:
cd ~/Android/Flash/anykernel
zip -r9 flash-kernel.zip * -x README flash-kernel.zip
Using the above commands, your newly completed kernel will be located at ~/Android/Flash/anykernel/flash-kernel.zip
If you followed all of the above steps successfully, congratulations, you built the kernel!! Once you have done all five steps, you will just need to follow steps 4 and 5 to update in the future. Below, I have linked a script that will do all of this for you (with some variable edits at the top); however, please do it manually first so you know exactly what you are doing enjoy!
https://raw.githubusercontent.com/nathanchance/angler/release/build-flash.sh
Below are screenshots of me doing this process in a virtual machine (Mint 18) with 2 cores, 4GB of RAM, and 15GB of storage on my MacBook with a 2.5 GHz Intel Core i5 and 8GB of RAM (fairly average in the world of PC specs I think). It's totally possible to do this on your own. Additionally, in the next post, I have provided some tips for customizing the kernel even further!
How to do your own modifications to the kernel
Introduction
When I was getting started with compiling kernels, I relied a lot on Google and git documentation (this website is the best) to get what I wanted to accomplished. As a lot of that reading can be kind of dense. I decided to boil it down to the few commands and procedures you will need to modify this kernel (or any git repo).
LEARN GIT FROM THE COMMAND LINE. Seriously, do not use a program like SmartGit, it will cripple you and your workflow and potentially screw up authorship. Do it the right way using this guide.
Note: This is NOT supposed to be a super technical guide/how-to on Git. If there is something that is glaringly inaccurate, I will fix it but I am not going to add a ton of extra text here as there are plenty of Git tutorials you can easily find with Google. This is a crash course.
Git Introduction
What's a git?
Git is a version control system that allows developers to keep track of their changes to code. This is extremely helpful when working with large amount of code and changes across a large range of files (such as a kernel or ROM) as it allows you to easily revert or add changes as you see fit.
Terminology
These are not super precise definitions (you can find technical ones by Googling "what is a <word> git"):
Repository: the location that holds all of your code changes and files (this can be local or remote).
Commit: a change to your code
Branch: different snapshots of your code
Cloning: copying a remote repo to a local one
Forking: copying a repo (either remote or local) and making changes to it without affecting the other
Reading Github
Github is the most popular site for hosting git repositories. This is what my repo is hosted, which I will be using for my example. If you go to https://github.com/nathanchance/Ninja-Kernel, you will be greeted with this screen:
Here we will unpack those terms above. The repository (that view) contains files. At the top, you will see a tab called Commits. Clicking on this will bring you to a view similar to this:
This is a full list of the changes to the repository. The name of the commit message generally describes what it does and clicking on the title of the commit will bring you into the detailed view:
In this commit, I have renamed the version from 1.1.1 to 1.2. Some commits will be very small like this, others won't be. Additionally, in the righthand corner of the commit message, you'll see a commit hash (bunch of numbers and letters). That will come in handy later.
Back on the home page of the repo, if you click on the branch dropdown, you will see the various different branches of the repo:
If you click on a branch (let's do n in this case), you'll be brought to a page similar to the home page but there may be different commits in the commits tab. In this case, I have committed version 2.1 instead of 1.1:
This should help you navigate around Github and find commits you may want to merge.
Git Documentation
Throughout the next sections, I will be using commands to show you how to modify the source. This section goes over the command format, what it does, and an example.
git clone
Format:
Code:
# Clone a repo into a directory the same as its remote name
git clone <url>
# Clone a repo into a directory with a specified name
git clone <url> <name_of_folder>
What it does:
- Copies a remote repo into a folder on your computer
Example:
Code:
git clone https://github.com/nathanchance/Ninja-Kernel.git
git clone https://github.com/nathanchance/Ninja-Kernel.git Ninja
Screenshot:
git checkout
Format:
Code:
# Changes to the specified branch (if it does not exist, it creates one)
git checkout <branch>
# Creates a new branch based off the specified one
git checkout -b <branch> <branch_to_be_based_on>
What it does:
- Whenever you want to make changes to files, you need to make sure you are on the right branch. You can see what branch you are currently on by running git branch; if you are on a different one, switch to the right one using the above commands
- If you want to create a new branch, use the second command.
Example:
Code:
# Switches to the m branch
git checkout m
# Creates a new branch "new-branch" based on old-branch
git checkout -b new-branch old-branch
Screenshot:
git add
Format:
Code:
# Adds a file to the staging index
git add <file>
# Adds all of the changed files to the staging index
git add -a
What it does:
- This adds a file to the list of changes to commit (staging). So let's say you modify the README of your repo, deleting a line, and adding one. In order to register that change, you need to add the file to your staging index.
Example:
Code:
# Adds the file README to the index
git add README
Screenshot:
git status
Format:
Code:
# Tells you what the status is of your repo
git status
What it does:
- Allows you to see what files you have staged for commit, where you are relative to where you are with your remote
Example:
Code:
Screenshot:
git commit
Format:
Code:
# Commit your changes and bring up a nano window to write your message
git commit
# Commit your changes and create the message as a parameter
git commit -m "Commit message here"
What it does:
- Saves all of your staged changes in a commit
Example:
Code:
git commit -m "This is a test commit message"
Screenshot:
git revert
Format:
Code:
# Undoes the changes by a certain commit
git revert <hash>
What it does:
- Undoes the commit you have chosen.
- NOTE: It is a good idea to add to the commit message why you are reverting a commit so people don't have to guess (did you not like it or did it break something?)
Example:
Code:
git revert 18d72d42648de9fde9bedcc810a936e0d253af9c
Screenshot:
git pull
Format:
Code:
git pull
What it does:
- Retrieves the latest changes from your remote repository (like Github)
Screenshot:
git push
Format:
Code:
git push
What it does:
- Opposite of git pull, it sends your latest changes from your local repository to the remote
Screenshot:
git fetch
Format:
Code:
# Fetches the commits from the main branch of a remote repository
git fetch <url>
# Fetches the commits from the specified branch of a remote repository
git fetch <url> <branch>
What it does:
- Whenever you want to merge or cherry pick something from a repository other than your own, you need to fetch it so git knows where to pull commits from.
Example:
Code:
# Fetches the commits from the n branch of my repo
git fetch https://github.com/nathanchance/Ninja-Kernel.git n
Screenshot:
git merge
Format:
Code:
# Merge one branch of your repo with the current one
git merge <branch_to_merge>
# Merge a remote branch into the current one
git merge <remote>/<branch_to_merge>
What it does:
- Takes the commits in the specified branch and merges the ones that are not present in the current branch.
Example:
Code:
git merge n
git cherry-pick
Format:
Code:
# Takes a commit either from another branch or the list you fetched and merges it into your current branch
git cherry-pick <hash>
# Pick a range of commits
git cherry-pick <hash_1>^..<hash_2>
What it does:
- If you have a commit in another branch and want to merge it into your current branch, get the hash either from Github if pushed or git log (below) and run git cherry-pick
- If you have a commit in a remote repo you want to merge into your current branch, fetch the remote repo and run git cherry-pick with its hash (found on the remote repo's website)
Example:
Code:
git cherry-pick 68a8ff686ccb2ff01cbaa7b2704c4da6c066ecc8
gcp 7204eedd1076bd87c624aab4fb9c0cc6ac50933b^..102a72b895afb063240fb09292c17e11155e8dd9
Screenshot:
git log
Format:
Code:
# Brings up a list of commits with their full messages and hashes
git log
# Brings up a list of commits with the title of their message and full hashes
git lot --format="short"
# Brings up a list of commits with short hashes and messages
git log --oneline
What it does:
- This command allows you to see the revision history of your local repo
Screenshots: (git log, git log --format=short, then git log --oneline
How To's
Merge Android security updates
There are two ways to do this (I personally recommend cherry-picking)
1. Cherry pick the individual commits
1. Fetch Google's repo
Google's kernel repository is located here: https://android.googlesource.com/kernel/msm/
Currently, Angler is on the branch android-msm-angler-3.10-marshmallow-dr1.6-1
Thus, the command looks like this:
Code:
git fetch https://android.googlesource.com/kernel/msm/ android-msm-angler-3.10-marshmallow-dr1.6-1
2. Figure out which commits you need to pick
Click on the branch you want to merge commits from in Google's list
Click on the "log" link next to the commit hash
In that list, find the first commit at the top of the list and open it in a new tab (now referred to as the last commit)
Start skimming the list, looking for the next commit that has a green android-6.0.1_r# next to it
Open the commit right above that commit (now referred to as the first commit)
Copy the individual hashes of the first and last commit (make sure to know which is which!)
3. Pick the commits!
Code:
git cherry-pick <hash_of_first_commit>^..<hash_of_second_commit>
Example
Commit log located here: https://android.googlesource.com/kernel/msm/+log/android-msm-angler-3.10-marshmallow-dr1.6-1
First commit: https://android.googlesource.com/kernel/msm/+/46b9920417db45ee0a93f16a07e80666219446d3
Last commit: https://android.googlesource.com/kernel/msm/+/909746bb5e4be07344b4fe4ce95c57fbfc8d9595
Code:
git fetch https://android.googlesource.com/kernel/msm/ android-msm-angler-3.10-marshmallow-dr1.6-1
git cherry-pick 46b9920417db45ee0a93f16a07e80666219446d3^..909746bb5e4be07344b4fe4ce95c57fbfc8d9595
2. Merge the whole branch
1. Add a remote
Google's kernel repository is located here: https://android.googlesource.com/kernel/msm/. A remote allows us to easily fetch a repo we use often. To add a remote, follow the below format:
Code:
git remote add <name> <url>
In this case:
Code:
git remote add aosp https://android.googlesource.com/kernel/msm/
Example:
2. Fetch the repo
Format:
Code:
git fetch <name_of_remote_above>
Example:
Code:
git fetch aosp
3. Merge the branch
On the left of Google's website, you will notice there are some branches. Currently, Angler is on android-msm-angler-3.10-marshmallow-dr1.6-1 (you can tell by the commit date of the first commit)
The format looks like this:
Code:
git merge <name_of_remote>/<branch_to_merge>
Example:
Code:
git merge aosp/android-msm-angler-3.10-marshmallow-dr1.6-1
Merge new Linux kernel updates
1. Fetch the kernel.org repo
We are on the 3.10 branch of the stable repo: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git
Code:
git fetch git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux-3.10.y
2. Figure out which commits you need
Go to the kernel.org website
In the 3.10 line, click browse
You will be presented with a full list of commits
Open the first one in a new tab (which should be something in the form of "Linux 3.10.y" (now known as last commit)
Use the next link at the bottom of the list and skim through the list of commits until you see the next commit in the form of "Linux 3.10.y"
Open the commit directly above the previous commit (now known as the first commit)
Copy the hash of the first and last commits (making sure you know the difference!)
3. Pick the commits
Code:
git cherry-pick <hash_of_first_commit>^..<hash_of_last_commit>
Example
First commit of 3.10.102: https://git.kernel.org/cgit/linux/k.../?id=ce9588a047eae53baf1607a408a8d1d5363f5fde (found on this page)
Last commit of 3.10.102: https://git.kernel.org/cgit/linux/k.../?id=ca1199fccf14540e86f6da955333e31d6fec5f3e (found on this page)
Code:
git fetch git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux-3.10.y
git cherry-pick ce9588a047eae53baf1607a408a8d1d5363f5fde^..ca1199fccf14540e86f6da955333e31d6fec5f3e
If you get any conflicts. resolve them using the fix merge conflicts section below.
Add a feature from another kernel
I will be using @flar2's wake gestures as an example but this process will work for any set of commits. I have provided a Git example repo here. On the aosp-latest branch will be the latest kernel source from AOSP. If you want to follow along in this tutorial, clone that repo and checkout the branch like so:
Code:
git clone https://github.com/nathanchance/Git-Example.git && cd Git-Example && git checkout aosp-latest
1. Figure out which commits you need
This can be a little bit tricky. When adding a feature, there may be more than one commit you need to grab. The trick I use for figuring out if this is the case is going into the detailed commit view (take this commit for example), copying the location of one of the files changed, going back to the head of the repo (in this case, here), navigating to that file, and clicking the history button in the right hand corner. This should give you something like this:
Now copy all of the commit hashes (you can click the little arrow clipboard icon next to the commit message to do this automatically. That gives us this list for wake gestures:
12cea9ee691f9794d3b0a76c6b381b4bde670bda
f4286dee51fde5c1082732381c9474c8381b8fb6
4c9e7f3f63eaf7514154633d320a6d4dd4f94831
5d10751af74617c23dfaf93bdf8149e39ce6da41
79d2aae008e51ed32d64e8a4a1dcf03231a22439
6495a2b15e501ce81bb58d39edd5923d52c2e45f
37daa721971b17a01d503c6dfa66c7c93b7cde2d
2. Fetch the repo
Now we need to fetch the repo that has the commits we need (in this case, the ElementalX repo).
Code:
git fetch https://github.com/flar2/angler.git ElementalX-1.00
Example:
3. Pick the commits
After fetching the repos, make sure you are on the proper branch using git checkout and git branch. Once this looks good, run git cherry-pick with the hashes:
Code:
git cherry-pick 12cea9ee691f9794d3b0a76c6b381b4bde670bda
git cherry-pick f4286dee51fde5c1082732381c9474c8381b8fb6
git cherry-pick 4c9e7f3f63eaf7514154633d320a6d4dd4f94831
git cherry-pick 5d10751af74617c23dfaf93bdf8149e39ce6da41
git cherry-pick 79d2aae008e51ed32d64e8a4a1dcf03231a22439
git cherry-pick 6495a2b15e501ce81bb58d39edd5923d52c2e45f
git cherry-pick 37daa721971b17a01d503c6dfa66c7c93b7cde2d
Screenshot:
In this case, I was lucky not to have any conflicts. If you try to pick something and there are differences between the code that conflict, you'll get a merge conflict which you will need to fix before committing the cherry pick.
4. Check for any conflicts
Now when you merge a feature, you may need to enable in your defconfig. In this case, we need to add the line CONFIG_WAKE_GESTURES=y (tells the compiler to build the wake gestures module and add it). You can see an example of this here and here.
Fix merge conflicts
I'm not going to explain it any better than these guides
https://help.github.com/articles/resolving-a-merge-conflict-from-the-command-line/
https://githowto.com/resolving_conflicts
http://stackoverflow.com/questions/161813/how-to-resolve-merge-conflicts-in-git
Special note: Authorship
Authorship is very important in the open source world, as it is what allows people to determine who is responsible for the commit and receive the proper credit. The general rule of thumb is if you didn't think of the idea and write the code, you are not the author. If you are cherry-picking properly, authorship should be automatically maintained; however, if you have to commit something that isn't yours or do some other wonky stuff, you may become the author. No worries though, it is very easy to fix it!
Apply authorship during commit:
Format:
Code:
git commit --author="Full Name <[email protected]>"
Example:
Code:
git commit --author="Nathan Chancellor <[email protected]>"
Screenshots:
Apply authorship after commit
Format:
Code:
git commit --amend --author="Full Name <[email protected]>"
Example:
Code:
git commit --amend --author="Nathan Chancellor <[email protected]>"
Screenshots:
Note: if you have already pushed your commit, you will need to amend the commit and force push it to change it:
Code:
git push origin --force
Good to see you with a kernel of yours.
Should have done this earlier
Sent from my angler using XDA Labs
Congrats!
I found you a cover picture
Nice, let's test, thanks.
cool
Gesendet von meinem Nexus 6P mit Tapatalk
Cool ?
Envoyé de mon Nexus 6P en utilisant Tapatalk
Nice one buddy, been using the kernel now for a few hours and its stable and blazing fast.
Only thing I noticed and maybe you could look into this, is sometimes when I unlock the phone and I am using chill governor it gets stuck at 302 frequency and will not go up at all........ Tried it on other kernel with chill, running same setup and was not the case. Thanks for the awesome kernel, very snappy!
Thanks @nathanchance. Great to see your own kernel make a debut.
Glad to see you adding your own flavors and goodies and sharing the finished product with us!
Flashing this now!
Nice looking kernel dish! Kudo's
yung40oz84 said:
Only thing I noticed and maybe you could look into this, is sometimes when I unlock the phone and I am using chill governor it gets stuck at 302 frequency and will not go up at all........ Tried it on other kernel with chill, running same setup and was not the case. Thanks for the awesome kernel, very snappy!
Click to expand...
Click to collapse
What kernel had it working properly? I am going to guess it is a wakeup issue or something. That governor is based on conservative, which has not be updated (at least on our Linux versiom 3.10.x) for big.LITTLE, so I am a little surprised it is working well elsewhere.
I went with the 'tried & true' settings, and immediately noticed ui was snappier. Only been running it half a day so I expect things to get better as it settles.
Nice work!
Added to Nexus 6P index thread:
[INDEX] Huawei Nexus 6P
Just flashed this along with PN, excited to give it a run. Thanks for the kernel
Flashed it along with beltz rom and so far I'm very satisfied! Thank you.
Sent from my Nexus 6P using XDA-Developers mobile app
Best kernel for 6P. Not even an opinion, just based on performance.
I run Arise always and usually i need to put min freq of clust 1 to 480 MHz, so theres no clipping. This is with other kernels.
Ninja runs it perfect and smooth even with min freq set to 302 MHZ.
Whilst compiling Cyanogenmod I got this error:
Code:
ccache: FATAL: Could not find compiler "arm-eabi-gcc" in PATH.
Based on the manifest
https://github.com/LineageOS/android/blob/cm-12.1/default.xml#L542
the compiler is platform/prebuilts/gcc/linux-x86/arm/arm-eabi-4.8
from
https://android.googlesource.com/pl...inux-x86/arm/arm-eabi-4.8/+/android-5.1.1_r37
If I add that full path to my PATH variable I can compile the ROM successfully.
However, that seems wrong to me.
The compiler is already part of the source tree.
Shouldn't something in one of the make files or build scripts be setting this rather than me setting it manually?
Note: I am not using a custom toolchain, it is the standard one from aosp.
opticyclic said:
Whilst compiling Cyanogenmod I got this error:
Code:
ccache: FATAL: Could not find compiler "arm-eabi-gcc" in PATH.
Based on the manifest
https://github.com/LineageOS/android/blob/cm-12.1/default.xml#L542
the compiler is platform/prebuilts/gcc/linux-x86/arm/arm-eabi-4.8
from
https://android.googlesource.com/pl...inux-x86/arm/arm-eabi-4.8/+/android-5.1.1_r37
If I add that full path to my PATH variable I can compile the ROM successfully.
However, that seems wrong to me.
The compiler is already part of the source tree.
Shouldn't something in one of the make files or build scripts be setting this rather than me setting it manually?
Note: I am not using a custom toolchain, it is the standard one from aosp.
Click to expand...
Click to collapse
Toolchains aren't automatically set, at least for now. Possible causes being...
There are multiple versions of Toolchain that are synced together, and it isn't necessary that the newer version of the same toolchain "must" compile the code with success, which was properly compiled by an older version.
Also, by this, the developer has complete freedom to change the ROM/Kernel toolchain.
Coming to the Question in Thread Title, If you're building a ROM, then toolchain has to be added in PATH
Arvind7352 said:
Toolchains aren't automatically set, at least for now. Possible causes being...
There are multiple versions of Toolchain that are synced together, and it isn't necessary that the newer version of the same toolchain "must" compile the code with success, which was properly compiled by an older version.
Also, by this, the developer has complete freedom to change the ROM/Kernel toolchain.
Coming to the Question in Thread Title, If you're building a ROM, then toolchain has to be added in PATH
Click to expand...
Click to collapse
Thanks for clarifying.
Ideally it should scan the source tree for the compiler by default so it has something to fall back to in case a custom one is not specified.
What confused me is that I don't see this nugget of information specified in any of the guides I read about compiling ROMs.
Hello!
This is a development thread for the project of upgrading of the Linux kernel for the U8500 platform.
Builds provided here (at the moment of writing this message) are not considered to be used as a daily driver, by any means, - these are rather a dev preview versions.
For now, there are a several LK builds (the highest currently supported kernel version is 3.10).
Because builds here are really not stable, I'll just leave a disclaimer here.
Code:
#include <std/disclaimer.h>
/*
* I am not responsible for bricked devices, dead SD cards, thermonuclear
* war, or the current economic crisis caused by you following these
* directions. YOU are choosing to make these modificiations, and
* if you point your finger at me for messing up your device, I will
* laugh at you.
*/
Working features
RIL
Camera (front & rear) - only works in <3.8
Video (playback & recording)
Audio (playback & recording)
Wifi
Bluetooth (broken in >=3.8, needs a workaround to manually startup)
USB, ADB
Tethering (not tested)
GPS (not tested)
Sensors
Known bugs
IRQs are mishandled by some device drivers (abb_btemp, abb_fg)
proximity sensor might not work (not tested, cause it's broken on my device)
deep sleep might not work at a times
MMC driver works unreliably in >=3.8 (contiguous usage might lead in a data corruption)
networking is not fully-functional (no mobile data)*
camera is broken in 3.8
*some other features of the android kernel might not present - it's because these kernels lacks android-specific patches.
Sources
LK 3.5
LK 3.6
LK 3.7
LK 3.8
LK 3.9
LK 3.10
Downloads
http://xda.mister-freeze.eu/XDA-files/ChronoMonochrome/misc/upgrading
Installation
install chrono kernel r5.2 or higher (this is needed to pick up the necessary scripts, incl. bootscripts, etc)
reboot to recovery
install build linked in "Downloads" section
Credits
Linux kernel development community
Google
ST-Ericcson
Samsung
Team Canjica
XDA:DevDB Information
U8500 Linux kernel upgrading project, Kernel for the Samsung Galaxy Ace II
Contributors
ChronoMonochrome
Kernel Special Features:
Version Information
Status: Alpha
Created 2017-05-09
Last Updated 2017-05-10
Reserved
Porting
The porting a higher kernel version tehnique I'll describe here is not intended to be a guide for dummies. I'll assume you've already built a kernel for your device, familiar with git versioning control usage and with some porting / coding tehniques.
Firstly, you need a cleaned source for your device. By "cleaned" I mean, there are no Linux incremental patches, android changes applied, manufacture-specific patches are avoided when possible and so on - you need sources as closest to a "pure" Linux kernel as possible. Otherwise you'll have later need to deal with conflicts resolution, you'll most likely be unable to resolve and the kernel won't boot.
So, without a further forewords, the tehnique is below:
1) As was previously mentioned, a clean kernel source is required, I'll assume we are starting from LK-3.4 ( https://github.com/ChronoMonochrome/Chrono_Kernel-1/commits/ea228ee0f5e9935841aff25c62fa163cd78dc01d ) and porting a higher kernel versions. A kernel base needs to be tested for any bugs just to distinguish, which bugs were intruduced during porting from those ones that already present in a kernel base.
2) The following steps will mostly use git bisect and git merge commands in order to merge all the changes from a higher kernel versions and help to find / resolve the bugs that were introduced. I suggest copying a git kernel repo that you use for building to a somewhere else, so you can use it , e.g. for grepping a different versions source, bisecting the revisions and so on, so don't need to bother messing up in your main repo that you use for build.
3) Firstly, lets just try to merge a higher kernel version, e.g. LK 3.5 by issuing a command git merge lk-3.5. You'll likely have a lot of merge conflicts, most of which you can resolve with resetting the paths to a some revision (either a kernel base - lk 3.4, or the version you do port, or just another suitable conflict resolution). So I suggest to write those paths to a text file, like so:
Code:
arch/arm/boot
arch/arm/mach-ux500
arch/arm/plat-nomadik
drivers/mmc
include/linux/mmc
drivers/usb
include/linux/usb
drivers/mfd
include/linux/mfd
...
Write all the paths you intend to reset to the kernel base, you most likely need to re-use them later. To actually perform a resetting source, you can issue
Code:
for path in $(cat file_with_a_paths.txt | xargs)
do
git checkout COMMIT $path
done
Be sure not to put to this file anything not the device-specific! Resetting to the kernel base should be avoided when possible (never ever try resetting archictecture-specific paths, e.g. arch/arm/kernel, arch/arm/mm and so on - unless you really know that kernel will boot thereafter, instead, you have to manually resolve such conflicts). Resolve any other conflicts by resetting paths to the porting source (e.g. LK 3.5).
Note. While resetting with a paths is probably not the most accurate tehnique, but people don't live that long to use more accurate approach, e.g. performing git cherry-pick for every upstream commit and then manually resolving all the conflicts, you'll just sooner or later get bothered and will abandon it.
4) When you're done with the previous steps you can try building kernel. You'll likely have a build errors - because some part of a source got not updated (e.g. the device-specific drivers), you need manually implement the necessary by a higher kernel version changes. Firstly check if an upstream kernel contains the necessary fixes (example: https://github.com/ChronoMonochrome/Chrono_Kernel-1/commit/9fae8c449b710f502662da1cbcf26ece5a098af9 , https://github.com/ChronoMonochrome/Chrono_Kernel-1/commit/fe027c25d6db0d100937deb5248e249ec5b24ee7 ). If the driver you are porting doesn't exist in the upstream, you can also try to find a similar change and mimic it: https://github.com/ChronoMonochrome/Chrono_Kernel-1/commit/5f2e7afbf2ac3284dc62b3d96a0627c7f99ed4e9 ( ported similarly to https://github.com/ChronoMonochrome/Chrono_Kernel-1/commit/526c597 ). In the worst case scenario you will need to examine the upstream changes and apply the changes so that the drivers complies to the upstream changes: https://github.com/ChronoMonochrome/Chrono_Kernel-1/commit/ea6432d167 .
5) If everything is done properly and you're lucky enough, the compiled kernel might already bootup. If not, you'll need to find a culprint that doesn't let the device to boot up. Switch to a copy of your kernel sources, reset the source to the base kernel version (e.g. LK 3.4), issue git bisect good, then issue git bisect bad lk-3.5, git will reset to a somewhere in a middle between of LK 3.4 and LK 3.5.
6) Save your changes in the kernel repo, by assigning a some branch to it, switch to the source base, merge all the fixes you've already introduced, then merge the revision you have got in the previous step by bisecting the tree. Repeat these steps until you'll find a first bad commit.
7) If you are already on this step, the most trickiest part starts here - testing (hopefully) working kernel for bugs (if any). While logs can be useful sometimes (so you can google the failing messages and find something useful), there are also many bugs you can find only performing git bisect tehnique decribed above.
The decribed algorithm only possible thanks to having a clean kernel source. The usage of this guide is not limited only to the kernel porting, it can be used on other projects as well, this is just what I've come across to, when I've ever started porting Linux kernel versions higher than LK3.4.
Reserved
I wonder if any of this expertise couldn't look pretty cool here too.
Wooooowwwewe
Oooh
Look whose good boys have been trying to win the STE mastermind prize as of lately
https://github.com/novathor-mainline/linux
https://git.kernel.org/pub/scm/linu...inux-nomadik.git/log/?h=ux500-skomer-v5.5-rc1
mirhl said:
Oooh
Look whose good boys have been trying to win the STE mastermind prize as of lately
https://github.com/novathor-mainline/linux
https://git.kernel.org/pub/scm/linu...inux-nomadik.git/log/?h=ux500-skomer-v5.5-rc1
Click to expand...
Click to collapse
Seriously!
mirhl said:
Oooh
Look whose good boys have been trying to win the STE mastermind prize as of lately
https://github.com/novathor-mainline/linux
https://git.kernel.org/pub/scm/linu...inux-nomadik.git/log/?h=ux500-skomer-v5.5-rc1
Click to expand...
Click to collapse
Wow, that's incredible
Exynos4412 already got some mainline support, it would be very nice to have this one supported too.
Aaaaand it's done, kinda.
ST-Ericsson NovaThor U8500 - postmarketOS
wiki.postmarketos.org
device/testing/linux-postmarketos-stericsson · master · postmarketOS / pmaports · GitLab
postmarketOS package build recipes
gitlab.com
Introduction
I am writing this guide as I was unable to find a well-documented thread over XDA (there are just a lot of them). In case there is anything that concerns this guide, feel free to ask in this thread. I expect you to know the basics of Linux, GIT, Android. Please look over some guides in the forum in case you are not familiar with them. In case you want to contribute to this, feel free to do so. I welcome everyone's contribution to be it fixing my derp, pointing a mistake or adding a new section.
Links to some nice documentations:
Git Documentation
Linux Kernel Newbies
How to take logs in Android
How to get an Android kernel up to date with linux-stable
Checking Current CAF Tag of the Kernel
We will be using best_kernel for this purpose, whose aim is to compare your current kernel source code with different CAF tags and select the best possible match using git diff.
Clone/Download your kernel repo and commit your changes(if not already committed) as best_kernel depends upon git diff to compare changes.
Depending upon your kernel version, add required CAF kernel repo as remote and fetch it. best_kernel will use this fetched data to compare changes.
After fetch is done, run best_kernel. It should run automatically if you have placed dotfiles in the $BIN directory and have the necessary dependencies installed.
best_kernel will compare changes and will provide results based on that. However, depending on the number of tags, it will take time. A lot of time, so go and grab a cup of coffee till then.
A Dummy Example:
Downloading stock kernel source code from OEM’s website
Code:
wget http://nokiaphones-opensource.azureedge.net/download/phones/Nokia6.1_V2.22J.tar.bz2 && tar xvjf Nokia6.1_V2.22J.tar.bz2 && cd kernel
Adding and committing all files due to lack of a .git dir.
Code:
git init && git add --all && git commit -m "Initial Commit"
As the kernel source code version is 4.4, I will add 4.4 remotes from CAF and fetch it.
Code:
git remote add caf https://source.codeaurora.org/quic/la/kernel/msm-4.4/ && git fetch caf
After fetch, I will check for the current CAF tag using best_kernel. As my device platform is sdm660 and CAF has been releasing sdm660 tags as sdm660.0, I will be using it as an argument in best_kernel. This will ensure that best_kernel doesn’t waste time checking CAF Tags of other platforms and will speed up the process considerably.
Code:
best_kernel "*sdm660.0"
RESULT will be something like this:
Code:
Best match TAG: LA.UM.6.2.r1-06900-sdm660.0 Lines changed: 177924
What is CAF/Code Aurora | CodeAurora Android Releases Page | CAF Kernel Sources
Creating/Importing GIT History for Kernel Source
A lot of OEMs release kernel sources in a compressed format excluding the .git dir which results in no git history. Creating or it's better to say to import one is simple and easy.
Download and check the current CAF tag of your Android kernel source.
Note: If your chipset is not Qualcomm, you should visit Linux Kernel Source or AOSP's Kernel Common and use that as a base while checking out to the specific version of your kernel source you are going to copy-paste. Qualcomm devices can also use that as a ref but I will recommend them to use CAF source.
Based on that CAF tag, download the respective kernel repository from CAF and checkout for that specific tag.
Copy and paste your downloaded kernel source (with no history) over the CAF one (which has the history of that CAF tag).
Commit the changes.
You have accomplished creating/importing history.
A Dummy Example:
My current CAF tag is LA.UM.6.2.r1-06900-sdm660.0.
Therefore, cloning kernel source from CAF and checking out for this tag while creating a new branch named rebase.
Code:
git clone https://source.codeaurora.org/quic/la/kernel/msm-4.4/
cd msm-4.4/
git checkout tags/LA.UM.6.2.r1-06900-sdm660.0 -b rebase
Now, I will copy-paste the kernel source that I downloaded from OEM's website over the CAF one. Assuming the downloaded kernel is in ~/Downloads/kernel and CAF kernel source is in ~/msm-4.4 directories.
Code:
#Renaming the dir so, that it will get merged while copying due to similar names
mv kernel/ msm-4.4/
cp -avr msm-4.4/ ~/msm-4.4
cd ~/msm-4.4
git status
git commit -m "Import Nokia Kernel Source"
Keep in mind that a .gitignore is generated which can and will ignore files during commit which were not getting ignored earlier due to its absence. In case during compilation you encounter any error stating about missing file, add it forcefully and commit again.
Remember, its always good to split that one big commit into small parts. It's helpful when you start rebasing your kernel usually when new CAF Tag for new android versions come out. Solving a few conflicts ina lot of commit is much better than solving a load of them in one. It also gives you a chance to easily diff on websites like GitHub so that you can figure what's a particular line is doing.
Doing that is also easy. Just reset head for a particular folder like driver/touchscreen and commit them separately. Check git rebase documentation which will help you with this.
Upstreaming your Kernel
Merging upstream versions is a really good thing. It brings in security patches, stability, new features and much more. The usual and best way is to visit Linux Kernel Source and check if there are upstream updates available. If yes, start merging them in your kernel source one by one.
Note: If your chipset is not Qualcomm, you should visit Linux Kernel Source or AOSP's Kernel Common and use that as a base. Qualcomm devices can also use that as a ref but I will recommend them to use CAF source.
A Dummy Example:
Assuming my current kernel version is v4.4.205, I will fetch v4.4.206 from Linux Kernel Source and merge that. I can also fetch and merge different branches from CAF Source which are usually upstreamed with Linux-stable (for example, aosp/upstream-linux-4.4.y branch)
Code:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git v4.4.206
git merge FETCH_HEAD
Fix conflicts if you get some. What to keep and what not is a question that depends upon the device to device. Usually, you should compare local changes and upstream and try to figure out what could be the best way to fix it. I will advise to check the history of that file in upstream and see which commit caused that conflict and why. You can also check similar merges by other same kernel versions to see if they had the same conflict and how they fixed it.
I will also advise merging AOSP's kernel/common and new CAF upstream tags (Only QCOM devices should merge new CAF tags) regularly when available. The reason is pretty simple. There are commits that get merged in AOSP like which improves performance or other stuff which doesn't get merged in Linux Kernel Source. This way you can get the benefit from all sources. This, however, is not pretty easy and might result in a huge number of conflicts considering if OEM has modified the source a lot from upstream.
CAF also hosts f2fs and other file system changes on their servers. Merge it if you want and use it.
Compiling Standalone Kernel
Compiling a kernel standalone requires you to have a compatible toolchain like GCC or Clang. The best way to figure out which one suits your need is testing and debugging. The standard AOSP GCC toolchains are recommended for devices with old kernels, to begin with, while newer ones can use clang with backports or upstream merges. In case you want to compile with clang, checking out android-kernel-clang is a good way, to begin with.
Compiling standalone kernels can give a lot of missing headers errors/warnings. This is due to not compiling it in a ROM environment. This can be easily fixed either by correcting paths of those header files being included or creating an out dir manually and using it as output. You can also use this commit to do that automatically.
Clone your kernel source and desired toolchain(s). Export arch and cross compiler arguments. Choose the desired defconfig to make and compile.
A Dummy Example:
I will clone the kernel source and toolchain in the same dir and export the following arguments in kernel source root dir.
Code:
export ARCH=arm64
export CROSS_COMPILE=/media/hdd/aayush/kernel/aarch64-elf-gcc/bin/aarch64-elf-
make nokia_defconfig
make -j$(nproc --all)
Kernel conflicts might occur. You can search on GitHub to check if someone has fixed that already, otherwise fix the issues yourself. Comparing code with upstream repositories is also a way to figure out the differences. Choosing defconfig can be a bit tricky if you are doing bringups. I recommend pulling the current defconfig from your device itself. Its usually by the name of config.gz in /proc/ of your device.
Links to some cross compiler
GNU Toolchain by ARM
DragonTC
GNU and Linaro
Working with TWRP from Scratch
This post is supposed to help you with building TWRP from scratch for the first time if you are a newbie. There are some good resources already available for that over the web, specially XDA. I am linking some good ones below that you can read for more info on it.
How to compile TWRP Touch Recovery
The aim of a device tree is to contain the device-specific code for the build you are doing. In this case, as we are building TWRP which is a custom recovery, only flags relating to building TWRP and a normal recovery are enough to build it. Adding system specific codes like partition sizes for the system, vendor, boot and other doesn't contribute anything. An example tree can be begonia's tree hosted on TeamWin's org maintained by me.
Now coming to the device you want to build TWRP for, you will the following :
- A working kernel
- Device-specific binaries/libraries etc required for features like encryption which heavily differs on OEM implementation
AB devices need an extra commit in their kernel to make 'fastboot boot' command work: init: initramfs: disable do_skip_initramfs . Additionally if your device ships a separate DTBO image you will also need the following flag and the DTBO image: begonia: Build DTBO image into recovery
Newer devices are nowadays coming with 'fastboot boot' command disabled. Nokia 8.1 is an example of it. To make it work again, you might need to tweak bootloader which can prove to be risky.
Now coming to creating a device tree for TWRP. The very first step is to look out if someone else has booted TWRP on a similar SOC, OEM or same device. It can be used as a reference to fix similar/OEM related bugs and hacks quickly. In case, one doesn't exist, just pick a latest one from Teamwin's Github org for the android version you are compiling for.
Reserved (2)
Wew many thanks,
Thanks
Have just started exploring xda chef central.
I cam across this guide, really ahlepful Hope you complete it soon.
Updated the thread. Fixed formatting, links and added a new section.
How To Unofficially Unlock Bootloader For LG G8
We all know LG G8 is a android device. It's open source. But I recently bought a LG G8 only but it's locked to sprint sim . No any sim work. I can't unlock bootloader. I can't root. I can't flag firmware or install custom recovery. Anyone know here , how to totally remove firmware and but another firmware ? Why we can't build a method to do that?
TheImpulson said:
Updated the thread. Fixed formatting, links and added a new section.
Click to expand...
Click to collapse
Can you create a guide for, "How to Rebase an Android Kernel". Like from Android 9 to Android 10?
thanks for this guide bro
While using best caf kernel, Prefix of the caf tag should start with a -
For example, "*- sdm660.0"
@theimpulson
Hey there,
Am getting this error del.dog/megupebinu.txt
While trying execute best_kernel script
This is helpful, thanks mate
@theimpulson I had a question: (Before reading this guide) I had already downloaded stock Samsung sources and git init push the thing to my Github. So will anything bad happen if I did it this way or do I have to rebase it with CAF?
Also what are the benefits of these so-called CAF based kernels everywhere? Improved performance? Cuz I really wanted to compile my own kernel specifically for performance with WSL-2 (shameless plug)
Thanks in advance and sorry if the @ bothered you,
Ishaq
NullCode said:
@theimpulson I had a question: (Before reading this guide) I had already downloaded stock Samsung sources and git init push the thing to my Github. So will anything bad happen if I did it this way or do I have to rebase it with CAF?
Also what are the benefits of these so-called CAF based kernels everywhere? Improved performance? Cuz I really wanted to compile my own kernel specifically for performance with WSL-2 (shameless plug)
Thanks in advance and sorry if the @ bothered you,
Ishaq
Click to expand...
Click to collapse
The existing sources are fine, no issues if you are fine without any git history.
SOC specific optimizations, bug fixes etc. Nothing more. So, yeah, if you want SOC specific optimizations, performance improvements, you will want to merge latest CAF tags for your kernel.
theimpulson said:
The existing sources are fine, no issues if you are fine without any git history.
SOC specific optimizations, bug fixes etc. Nothing more. So, yeah, if you want SOC specific optimizations, performance improvements, you will want to merge latest CAF tags for your kernel.
Click to expand...
Click to collapse
thanks a lot for the reply! Another thing, because of your suggestion I tried to Import the git history (and i succeeded). Then I wanted to upload these sources to my own Github, which I can't figure out how to do. It only gives Github repo for the place where it came from (msm-3.18). Help me please
Edit: The guides which I found for merging CAF tags say that my base also has to be CAF (which it isn't) and this guide here basically does not work