Yesterday I discovered a very interesting source release from Samsung: GT-S5830G_GB_Opensource.zip
Normally, there is a shared Samsung source release for the msm7k range of devices (Ace, Mini, Callisto, Beni, Gio; there is also partial support included for my GT-I5500/Europa, but I adapted the source to properly support my phone). All of these devices usually depend on Samsung's proprietary FSR (Flex Sector Remapper) drivers in order to access the flash memory, which are taken from the stock ROMs, but our dependence on this driver locks us into the 2.6.35 kernel since we don't have access to the driver source.
The new source that I found for the GT-S5830G model, however, appears to contain modifications for the purpose of transitioning from Samsung's proprietary FSR driver to the open source MTD (via msm_nand) driver. If this can work correctly, using this driver would be much preferred over the proprietary Samsung stuff.
Here's the source that I uploaded to github: https://github.com/psyke83/android_kernel_samsung_msm/tree/purenand
Some observations that I have made, keeping in mind that I'm testing on my GT-I5500:
There are two separate configs for the cooper rev03 (Galaxy Ace): the standard defconfig that uses fsr/rfs, and a "purenand" config that uses the mtd/yaffs2 drivers instead of fsr/rfs. In other words, it's using the open drivers for flash access. Here is the diff: https://gist.github.com/3365123
The dpram driver (Samsung's driver for communication with baseband, used by RIL) is patched to support MTD instead of BML when the proper config is set.
The drivers/mtd/devices/msm_nand.c driver is modified by Samsung, but they applied their patches to an older revision of msm_nand.c from Froyo. Here is the diff when comparing this file vs the Froyo revision, so you can see more clearly the changes: https://gist.github.com/3365161
By default, the msm_nand.c driver causes the kernel to hang on my device (this is true for both this source and the older 2.6.35 Samsung source not based on purenand). I have isolated the hang to the flash_onfi_probe function.
As you can see here, Samsung added code to bypass this function on the Cooper board, and use the secondary detection method only. If I include my board to this ifdef block, it solves the issue with the kernel hanging on my device. I also need to patch some checks in the onenand detection, because the driver explicitly looks for onenand devices with a device_id of 0x40 and num_of_buffers as 0x201, but the chip on the GT-I5500 is different (device_id is 0x50, num_of_buffers is 0x101). This patch solves these problems: https://gist.github.com/3365222
Here is a dmesg log from my device after patching the code: https://gist.github.com/3360727. For comparison purposes, look at the block mapping that the fsr driver reports for my device when using the BML mapping: https://github.com/psyke83/android_device_samsung_galaxy5/blob/gingerbread/BoardConfig.mk#L53
As you can see, the partitions names and order detection is correct for the msm_nand driver, but the address mappings are exactly half of what they are supposed to be (e.g. the first partition, mibib, should range from 0x00000000-0x00180000, but the mtd driver detects the memory range as 0x00000000-0x000C0000.
If I try to perform a "nandump -f /sdcard/cache.img /dev/mtd/mtd13", there are no obvious errors in the dmesg log, but the tool will dump the cache partition until the sd card becomes full (over 300mb in my case, but the real /cache partition is only 25MB), and will then output "nanddump: short write". The resulting dump will be filled with 0xFF when examined with a hex editor (even though I'm sure that the /cache partition is not blank in reality).
These are my findings so far. I'd appreciate any kernel hackers to help me out. If we can crack this problem and get open onenand drivers working, then our devices will no longer be locked to any specific kernel release. If you know any developers for the Samsung devices I mentioned at the beginning of the post, or anyone else who may be able to help, please direct them to this thread. Thanks!
Reserved for future use
I have also posted this topic on the MadTeam forum, and posted a lot of further details. Please see: http://madteam.co/forum/development-8/(devs-only)-mtdonenand-driver-development/
Post this on General Discussion, this section is inactive
Related
Hello,
My question is, is it possible to build the linux kernel for an android device without the usb gadget driver built-in? My motivation and what I have tried thus far is listed below.
I am attempting to emulate the behavior of connecting and disconnecting an android phone from a given host. I have looked through many methods to try to emulate this behavior, and the solution I am working on now is:
1. build a kernel WITHOUT support for usb gadget
2. build a kernel module that supports usb gadget
3. put together an application to insmod/rmmod the resulting kernel module
I have done all three parts, and now I am seeing kernel panics. In particular there are two actions that result in kernel panics
1. Plug android device WITHOUT usb gadget support into a host (the enumeration must be causing the panic)
2. insmod and rmmod the kernel module, then insmod it again (kernel module name is android.ko and the resulting kernel log messages that happen just before panic are below)
[ 257.163085] sysfs: cannot create duplicate filename '/devices/platform/omap/musb-omap2430/musb-hdrc/gadget/lun0'
[ 257.163177] Modules linked in: android(+) [last unloaded: android]
I am working with the Samsung Galaxy Nexus. I have verified the kernel building process and the module building process by making a stock kernel and a simple kernel module. So the issue IS particularly with the usb gadget driver module and/or the kernel without built-in android gadget usb.
I have also used a Ellysis USB analyzer to see at what point in enumeration the device is crashing. Unfortunately, enumeration does not even begin without the built-in usb gadget module.
Best,
Ryan
This is just a compilation about the topic and i have made it as much as noob friendly as i can. i will write it part wise when i get time. Hope it helps the community. if i missed something or wrote something wrong please point me out.
Do not forget to hit thanks if this helped
Linux Kernel and its Structure
PART I
Introduction:-
The Linux Kernel is as we all know is on of the most prominent example of free and open source software. Linux kernel is one of the most popular operating system kernel widely used by Linux distributions and variety of mobile devices. If was the result of a hobby of Finnish student Linus Torvalds and in 1991 he gave the birth to the Linux Kernel which redefined the concept of free and open source software.
The Linux Kernel is released under GNU General Public License version 2(GPLv2) and due to which more then a thousand people contribute to each kernel release. The large and dynamic developer and user community around Linux helps it to rapidly accumulate developers and users to adopt code from other free software projects to use with the Linux Kernel.
The main features of Linux Kernel are :-
1.Multitasking: several programs running at the same time.
2.Multiuser: several users on the same machine at the same time (and license for two users)
3.Scalibility: can run on super computers as well as tiny devices.
4. Multithreading: has native kernel support for multiple independent threads of control within a single process memory space and even has memory protection between processes, so that one program can't bring the whole system down.
demand loads executable: Linux only reads from disk those parts of a program that are actually used.
5.Free and open source:- All source code is available, including the whole kernel and all drivers, the development tools and all user programs; also, all of it can be freely distributed.
6.Multiple virtual consoles: several independent login sessions through the console, you switch by pressing a hot-key combination.
7.Supports several common filesystems, including minix, Xenix, and all the common system V filesystems, and has an advanced filesystem of its own.
8.An extensive networking support.
9.Modulatiry: Loads only those modules that are needed by the system, even at runtime.
10 Easy to code: Almost all the resources are available free, so users and developers can learn form existing codes already contributed by the community.
Defining System Calls:-
The main interface between the Linux Kernel and the user space is a set of system calls. Normally the system provides a library or API such as glibc, that provides the wrapper functions for the system calls. Kernel services like file operations, networking operations inter-process communications, threads, memory mapping etc are provided by over 300 system calls.
At present the system call interface is stable and only new system calls can be added by the Kernel developers. The user space applications usually never make a system call directly but rather it uses the corresponding C library function.
Getting Linux Kernel Sources:-
The Linux kernel Sources are free and open source as it is distributed by the GNU General Public License v2, you can get the source as released by Linus Torvalds at
HTML:
https://www.kernel.org
There are various versions available namely Prepatch, Minline, Stable, Longterm depending on the maintainers and stability status.
Thought many of the chip vendors supply their own kernel sources as you already have seen in case of Google device kernels maintained and distributed by Google. There are many sub-communities that maintain their own kernel, with usually new features or specific features, and my be stable or unstable.
The Linux Kernel sources can be downloaded at
HTML:
https://kernel.org/pub/linux/kernel/{version}/{branch}
as a full tarballs and patches.
But you can also get the kernel using the git version control system. Detailed description of the git version control system can be found here -
HTML:
http://git-scm.com/
The Linux kernel size vary from 500Mb to around 1Gb but you can get the kernel as archive in gzip, bzip2, xz depending upon the compression rate. The Linux core is pretty small but it contains thousands of device drivers, network protocol, many architectures, filesystems, documentations, firmware, scripts etc, so the large size of the Linux kernel.
The Linux Kernel is implemented in C as like all the UNIX system. A little assembly language is also used, but no C++ is used in the Linux Kernel. All the codes are compiled by gcc and a few alternative compilers, but ANSI C compile cannot compile the Linux Kernel as gcc specific extensions are used in the kernel code.
The main feature of the Linux kernel is that it has to be standalone, which means no user space code or part can be used in the Linux Kernel. The user space is then implemented on the top of the Linux Kernel. The Linux Kernel has to supply its own library implementations, and no standard C library functions are used in the kernel. But it provides its own similar C library.
The Linux Kernel is designed to be portable, and the kernel provides macros and functions to abstract the architecture specific details. The Linux Kernel never uses floating point unit as some of the architecture may not be compatible with it. The internal kernel API is not stable and can undergo changes between releases and so if you have to use the API then keep track of all the API available for the specific kernel release. An out-of-the tree compiled driver for a older version of the kernel may not work on the recent ones. Though the kernel to user space API does not change as it may break the existing scripts and program running in the user space.
GPL licensing and open source drivers:-
As mentioned earlier the Linux Kernel is licensed under the GNU General Public License version 2, which gives you the right to study, use, modify and share it without any cost. However it states that if the code is redistributed or shared modified or unmodified then it will be under the same license. More information about the GPLv2 can be found at -
HTML:
http://www.gnu.org/licenses/gpl-2.0.html
Thought there are a lot of propriety drivers that are available for different hardware. You cannot distribute a binary kernel that includes statistically compiled propriety drivers. Most of the common form of the propriety drivers can be found in the case of graphics cards, though open source drivers are also available for the same.
The main advantage of GPL licensed drivers is that you do not have to write the whole driver from scratch and you can reuse the code for similar free software. The GPL licensed drivers can be easily distributed inside a compiled kernel for example a Linux distribution. Free community contribution, support, testing can be attained. And the code becomes much more stable as many people review the code and contribute to it. Many device driver codes are also implemented in the user space. This is possible as the Linux Kernel provides a mechanism that allows the user space application to access the hardware directly. Certain drivers are handled partly in kernel space and partly in user space.
The user space drivers have a lot of advantages as you need no kernel coding skills to write a user space driver. User space drivers can be written in any language and they can even be kept propriety. The driver code can be killed and debugged without crashing the kernel, and moreover it can be swapped out for a much stable version without changing the kernel.
Part II
Part II
Overview Linux Kernel Source:-
The Linux Source structure is defined in this section. The directories give here are at the top level of the Linux source tree.
Makefile
This is the top level Makefile for the whole source tree. It defines a lot of useful variable and rules, such as the default gcc compilation flag. Here also we set the architecture and the version.
Documentation/
This directory contains a lot of useful information about configuring the kernel and all the help items. The help entries corresponding to the different configuration cannot be found here. They are located in the Kconfig files in each source directory. If you are starting out this folder you should now miss.
arch/
All the architecture corresponding codes are in this directory. Each architecture has its own directory underneath this directory. Some of the important sub-directories are -
Code:
arch/<ARCH>/mach-<machine>
machine/board specific code
Code:
arch/<ARCH>/include/asm
architecture specific headers
Code:
arch/<ARCH>boot/dts
Device tree architecture
block/
Block layer code
COPYING
This contains the license(GNU GPLv2)
CREDITS
This contains the main contributors are mentioned here.
crypto/
This is the cryptographic API for the use by the kernel itself
drivers/
In this directory we find all the device drivers of the peripheral devices. Only sound device driver is not located here.
firmware/
Firmware images extracted from old drivers are located here.
fs/
This directory contains the generic filesystem code, known as Virtual File System and also the code for each different filesystem are found in this directory. If you are on a Linux distribution, the root filesystem is probably ext4 filesystem. The code to read the ext4 filesystem can be found in
Code:
fs/ext4
.
include/
This directory contains all the kernel headers. During compilation a symbolic link from the asm-<ARCH> is created so the
Code:
#include <asm/file.h>
will get proper file for that architecture. There are other non-architectural specific header file.
init/
This directory contains the files main.c and version.c and code for creating an “early userspace”. The version.c defines the Linux version string.
ipc/
IPC stands for inter process communication. It contains the code for shared memory and other forms of process communications.
Kbuild
Part of the kernel build system.
kernel/
It contains the general kernel level code.
lib/
Routines of generic usefulness to all kernel code are here.
mm/
High level memory management codes are found here. Virtual memory(VM) implementation and early boot memory management are done here, as well as memory mapping of files, management of paging cache, memory allocation are done here.
net/
The high-level networking code is here. The low-level networking driver pass received packets up to and get get packets to send from this level, then it passes the data to the main application.
scripts/
This directory contains scripts that are useful in building the kernel, but does not include any code that is incorporated into the kernel itself.
sound/
Drivers for sound cards and other sound related code is placed here.
usr/
This directory contains code that builds a cpio-format archive containing a root filesystem image, which will be used for early user space.
tools/
Code for various user space tools
security/
Code for different Linux security models can be found here.
samples/
Sample codes
What is Kexec?
"Kexec", which is short for 'Kernel Execution' is derived from the Linux Kernel call "exec". It allows the "live" booting of a new kernel "over" the currently booted kernel without taking the device down for a reboot. This is extremely useful on locked bootloader devices, as a user with root authentication can boot a custom kernel without rebooting, and undergoing the security checks enforced by the bootloader. On unlocked devices, it can be used to "multi-boot" kernels on a device without requiring the kernels to be installed to the /boot partition.
Whilst Kexec is extremely useful, it also can be extremely hard to implement, as it needs to take all devices down, and bring them back up along with the new kernel, this can lead to some serious bugs, like devices not working after soft-boot, kernel corruption, device hangs, etc. This make it very device specific, and hard to get fully working, as it requires retrieving kernel crash logs, (often) UART serial output, and a ton of debugging.
What about this whole "Hardboot" thing?
The solution to this was written (initially) by Mike Kassick, who had the idea to "Hardboot" a kernel. Which is when a kernel is loaded into memory, a flag is set, the device is taken down for a full reboot, then the flag is read out by the primary kernel very early in the boot sequence, at which point, the "primary" kernel directly loads the new "secondary" kernel/ramdisk/passes arguements/etc.
This is much easier to implement than the normal Kexec SysCall, as it jumps to the new kernel before most devices are initiated, and in doing this, we allow the secondary kernel to initialize all the devices on its own, and not have to worry about taking them down.
Many people unknowingly make use of Kexec in the form of MultiROM, so, today, I thought I would do a write up on how to use it in practice.
Necessary Components:
* Boot.img (alternatively, the zImage-dtb/ramdisk you want to use)
* Unmkbootimg
* Kexec Binary (can be found in your specific devices MultiROM zip)
* Kexec Hardboot enabled Kernel installed (most custom kernels have it)
* Root Access
Downloads:
All the Binaries I've cross compiled/found can be downloaded here: https://www.dropbox.com/sh/7g5jcofv8j2gwg9/AAA-2b-wLiHq2z0nCMIHSHooa?dl=0
All the Linux Binaries you'll want/need are here: https://www.dropbox.com/sh/qcho8bhaoi8cdkc/AACGvmIQlb_3I9OQtNMqIQwva?dl=0
If you use Windows/Mac, just find the binaries equivalents for your platform.
How to use it?
1. Take the aforementioned Kexec Binary, and place it in /system/bin using ADB or A File Explorer, granting it permissions drwxdr-xdr-x (or chmod 0755 it)
2. Over on your desktop, make sure you have Unmkbootimg in an Executable location, and that you've blessed them as executable (chmod 0755 filename). Then run
Code:
unmkbootimg /path/to/your/boot.img
This will dump a zImage (rename it to zImage-dtb now, for semantics sake), and a ramdisk, labeled initramfs.cpio.gz (Initial RAM File System, in the form of a cpio.gz archive).
Now, put the kernel and ramdisk in a folder on your SD Card via MTP/ADB Push, I called mine "kexecstuffs".
3. Now open a mobile terminal, or an ADB Shell, and run
Code:
su
cd /sdcard/PathToYourFolder/
kexec --load-hardboot zImage-dtb –initrd=initramfs.cpio.gz --mem-min=0x20000000 --command-line="$(cat /proc/cmdline)" --boardname=shamu –dtb
Now, lets dissect the different arguments we are passing to Kexec:
--load-hardboot = Tells Kexec to make use of the Kexec Hardboot kernel function, and take the device down for a full reboot as opposed to soft-booting, like that used in the standard Kexec Linux SysCall
zImage-dtb = Name of your kernel file
--initrd = Points to the ramdisk to be used when booting the new kernel, if not set, the current ramdisk in the /boot partition. Most archive types are supported.
--mem-min = A reasonable value in memory where the kernel is loaded, serves as space for Kexec to do its work
--command-line = What arguments are passed to the new kernel, using "$(cat /proc/cmdline)" allows you to pass the currently booted kernel's arguments to the new kernel, which is what we want in the case of Shamu
--dtb = Defines that the board makes use of an Appended Device Tree, can be passed without a value (which will rely on Tasssdar's “boardname” value), or can have a compressed DTB image as its value
--boardname = Tasssdar's way to handle different DTB styles, we just need to pass “shamu” to it, and it'll use our DTB style
Now that we have successfully loaded the kernel into memory, lets execute it!
4. In that same Mobile Terminal/ADB Shell, run:
Code:
kexec -e
Although this guide is for the Nexus 6 (shamu), it should work all devices supported bu MultiROM, or on any device with a kernel that supports Kexec/Kexec Hardboot.
I hope this helped you to better understand what Kexec is/how to use it.
Hello! I can't write in Soft Development section, so I'm writing here.
I need to make a build of any Android-x86 with HCE ability. I tried to modify common Android-x86 and CM builds but did not succeed. Now I'm experimenting with Bliss-x86. The problem is that even when I define NFC service to be built into the image it doesn't appear in the OS.
So, what I have now:
- "external/libnfc-nci", "external/libnfc-nxp" and "packages/apps/Nfc" packages are defined as default in manifest.xml
- PRODUCT_PACKAGES parameter includes "Nfc" string in device/generic/common/device.mk and in packages.mk
- android.hardware.nfc.xml and android.hardware.nfc.hce.xml files are to be copied into the image
When I resync the android tree, I see that NFC related files are fetched from the server. But after the building I don't see /system/apps/Nfc.apk file and /data/data/com.android.nfc folder. NFC service is not starting on boot. I assume it's not compiled at all. Why that happens? The build process goes OK.
What exactly should I do to enable Nfcservice in my build?
Well, I solved the issue by defining "NfcNci" in device.mk (not "Nfc"). There also may be additional packets to include, e.g. "libnfc-nci", "libnfc-nxp", "libnfc_nci_jni", "Tag", "com.android.nfc_extras".
I have been trying to compile a userdebug kernel for months now and i finally got a kernel built but it would not boot. Well it booted but just to the splash screen.
This is a build for the LM-X210ULM. Im wanting to be able to use it a rooted phone to debug my other phones and possibly root them. I would also like to be able to build kernels and roms on my device which is prohibited in the user builds.
Ok the first issues i ran across after a non working kernel were LGE_USB_HOST_LIST_INPUT warnings which were easily taken care of by removing the items it said were missing dependencies.
Next are the SND_SOC_CONFIGS which i just removed the ITEMS listed as having missing dependencies.
But im still getting warnings about drivers/soc/qcom/kconfig which has something to do with fatal errorr and the value of the function being outside its normal paramaters. So i marked the defconfig line as not set for the error being fatal.
Now all of a sudden im getting fatal errors that stop my compilation having to do with the built-in.o file not being in the EDIDX out put and i grepped EDIDX and nothing was found
Please help me
i was wrong on grepping the exdid stuff but i did earlier and the only way i found to fix both errors was to go into the .config and unset arm_unwind.
I dont know what that will do to a working kernel but it compiles. Also i switched to the linux msm-3.18 source by caf since his toolchain works and gw? His source is less buggy,
But even after adding dependencies from the lg build i still am not producing a working image zImage or zImage-dtb
Ok i spent last night compiling ovet and over again. I compiled my kernel in about 8 different ways including using the defconfigs unaltered. But still i produce a Image zImage and zimage-dtb. I have renamed each image boot.img-kernel and repacked my boot.image and flashed it via fast boot. Each time it goes past the bootloader and hangs.
well here it is my actual 2nd compiled kernel for cv1 devices. Duhjokers laughter boot image with the upset clown kernel.
I figured out the non-booting issue problem kind of in a round about way. Part of it recompressing the boot.img after placing the kernel inside. For some reason fastboot is giving me a remote fail for anything 32mb. The kernel packed and patched by magisk is 34mb.
The other thing i think is that the compiler signs the kernel image as its being compiled but the boot.img itself will need some kind of signing itself so magisk comes into play there as tops patching system resigns the boot.img as it patches it.
Othet than that its good to go.
Features are set at
Fastcharge
Full OTG usb host and slave
performace rated w cpu governor
rcu boost
Smb
pre-empt
Handle panic
cma
Swap
auto wakelocks
loop device
Tun
Rndis
Rmnet
Spi
Pin control for msm devices
Devtmpfs
Ntfs
Ext4
ext2
ext3
vfat
Smb charging
Otg charging
Media and camera support
Confugured for
msm8916
msm8917
msm8920
msm8937
msm8940
msm8953
Debug android
Debug windows
Debug mass storage
Debug most generic gpio
Debug most generic ACM
Debug most generic
Debug usb gadget
Debug mutexes and spinlocks
Full encryption support and so much more
This baby is packed full of the latest features to help you debug most items around your house and full of the latest features for calling and internet. Enjoy
Im back and better than ever with the jokerfish kernel. Its killer!! SMILE
Ok i cant get force fast charge to show up in sys/kernel and its drivung me crazy. Huhuh.
Its compiling. I see the object files. i first used phy-msm-usb that wasnt showing up so retrofitted msm-otg into the mix per standard orrder back in the day but nothing. Every thing works all features are go but no fastcharge.
Could some one help me stop my compiling madness
Oh yea heres the flashable zip....