How to reconstruct a binary identical I9000XWJP6 kernel image, and more - Galaxy S I9000 Android Development

The idea of this exercise is (at least) to get a stable starting point for kernel development. The thing which is currently missing is a proper working .config. I have reconstructed it using differential analysis and in the process hoped to find which components have actually been activated and to uncover changes (or Easter eggs) in the sources which have not been advertised. Having a working and identical I9000XWJP6 kernel also means that open development can continue from the current official public release. From there the things possible are only limited by your imagination.
The following is a walk-through on how to build the kernel, description of pitfalls that will cause changes in .config to break, and some annotations on discoveries made in the process.
The things you need are:
Mandatory:
- I9000XWJP6 zImage : from your favorite location
- Source tree : opensource•samsung•com the GT-I9000 OpenSource Froyo Update JPM.zip
- Sourcery G++: www•codesourcery•com/sgpp/lite/arm/portal/release1039
- Tweak-Kit : <attached>
Optional:
- Arm enabled GCC and binutils, including a development libbfd.
- Lots of your favorite beverage
The md5sum of the zImage should be: 26e9d5d206baf1515144c6b8de6f10d2
It is critical that the Sourcery G++ version is 2009q3-67.
The Tweak-Kit contains the following components:
Readme.txt - You're reading it
mkvmlinux.cc - convert zImage to vmlinux and extract the init ramdisk image
I9000XWJP6_defconfig - default .config
stamp.patch - set date/time and such to original
style.patch - fix style related warnings
prototype.patch - fix prototype related warnings
error.patch - Recoverable errors
houston.patch - Unrecoverable errors
shadow.patch - Fixate data structures
I9000XWJP6.h - Fixated macros
I9000XWJP6.c - Entry point stubs
SOME THINGS I ENCOUNTERED IN THE PROCESS:
a) What I absolutely did not expect was that I found two different encodings of the build timestamp. I could deduce that the timezone was central Europe. I had the assumption it would be Asia or America.
b) What was to be expected is that the source tree is incomplete. The directories drivers/fsr and fs/rfs are missing. You can still compile the kernel as the missing files are used to build modules. Problems start when you change the config. Doing so will change entry points and data structures and your kernel might die a horrible death when it loads modules who are unaware on these changes. There is a workaround which I will explain later.
c) The weirdest thing I encountered were the functions enable_hlt() and disable_hlt(). The are located deep in the unwind tables, a section not intended for code. I spent many hours trying to figure out how they got there or why but I still have no clue. I found exactly only one way to reproduce this behaviour and it is certainly not due to a typo, accident or ignorance.
d) The kernel is not a production but a debug version. It has nearly all tracking/tracing/debug bells and whistles switched on. If the energy required to maintain the statistics where to emit light, you could use your Galaxy as a Christmas tree. Function profiling is enabled and has a considerable negative effect on performance, code is not optimized for size but speed, and unwind tables have been enabled which are not used. These have a really bad impact on footprint size. I really hope that the same compiler and settings are not used for the Android layer. Changing the config into a production version will not work (and crash) as the non-native modules expect the debugging hooks which will no longer exist. But the same workaround as above can be used.
e) The functionality of the power management domains have been optimized to oblivion due to the excessive placing of code disabling comments in large parts of the clock, power management and mach-aries.c. Maybe because the Galaxy hardware is too different than the evaluation boards, or the hardware is buggy and disabling the code makes it less unstable, or there was just not enough time to get the code working. Anyway, at this moment I have no oversight into what degree the absence of power domains influence battery usage.
f) When I started examining the binary code I was puzzled by snippets of code I could not reproduce. Even worse, I encountered snippets that were just questionable. Unusual instruction sequences, and resister usage. Thinking I bumped into a GCC bug, I started debugging the compiler and even tweaked instruction scheduling weights but with no satisfying outcome. I know that GCC is very stubborn with regard to saving and clobbering registers in/across function calls and the code I saw was just incorrect. I knew a different compiler was used and I suddenly realized that it may be more different than what first meets the eye. The culprit turned out to be Sourcery G++. It is a private maintained branch of GCC for reasons I have not investigated. Even the Sourcery assembler is tainted as it played a nasty trick on me with the enable_hlt/disable_hlt thing. I do not like the code I see and I am aiming into getting the sources stock GCC friendly with a working kernel. However, GCC and Sourcery generate code which seem difficult to mix, but I'm getting closer.
g) Compiler warnings. Many of the Samsung sources generate warnings, something I really dislike. In my opinion a warning is emitted for a piece of code which can be interpreted in several ways, leaving the compiler to choose which. Usually it will choose the wrong one. Most warnings were related to coding style shortcuts, a couple of incorrect function prototype resulting in functions that should return int to return random or falsely ignoring return values. There were also a couple of nasties like deference of uninitialized pointers, accessing out-of-bound data and mixing clock data-structures of different types. Included are a number of patches to fix them.
h) I looked deeper into why GCC and Sourcery won't mix and discovered that they have different implementations with regard to constant definition within enum declarations. Google points to the staring point "GCC bug 30260" where is written that the behaviour of enumeration constants has changed to becoming signed int. I have noticed that even explicit unsigned values will change to signed.
Here is an example of what is going wrong:
Take following declaration
Code:
enum rt_class_t { RT_TABLE_MAX=0xFFFFFFFF }
. GCC will consider RT_TABLE_MAX to be -1, and Sourcery will consider it 4294967295. Now, in net/ipv4/fib_rules.c there is this code snippet
Code:
for (u32 id = 1; id <= RT_TABLE_MAX; id++)
GCC will skip the loop, and Sourcery will have a hard time doing nothing.
There are more examples like calculating the location of physical memory or signed/unsigned comparisons. The compiler switches -fwrapv and -fstrict-overflow might influence things, but it general the behaviour is hardcoded and both compilers have a different flavour. I think it would be wiser to choose the GCC flavour as it is more widespread and thus better tested (and fixed).
i) GCC. I noticed that early versions of kernels compiled with GCC would not start. At first I thought it was because of Sourcery /GCC code generating differences. After a number of buxfixes (in error.patch) I suddenly noticed that the GCC kernel is working. My phone is running a GCC compiled production configured kernel for nearly a week.
j) "Houston, we've had a problem" with the light sensor. One of the compiler warnings brought me to the file drivers/sensor/optical/gp2a.c. There within are located two routines which read the light and proximity sensor. They seem copy-pasted identical, however the sensor value types are different as the proximity value is a char and the light intensity a double. What is more convenient than to simply change the data type of the supplied buffer in the function prototype. Now headache starts as the semantics of the read (and write) call say that the unit size is byte. So returning "1" indicates that only the first byte of the sensor value is copied. Also, there is no bounds/access checking so supplying an invalid pointer to the call will crash the kernel. So, assuming this is all one big mistake, I redesigned the function to do better (see houston.patch) and built a new kernel with it. To my utter surprise my battery charge extended from <24 hours to 2 days and 20 hours.
However... I also noticed that my backlight intensity level was constant at it's lowest although the setting was set to auto. I needed to know how the caller invokes the call, but after an extensive search of internet and android sources it is still something I have not found. Heuristics show that the reading the light sensor is called with a buffer length of 1, and the returned value is only accepted when returning a 1 and that the sensor value type is a double (8 bytes). This is wrong: read() semantics require that you supply a length of 8, and expect a return value of 8. This may be the base of many light sensor issues I found when Googling.
Anyway, I returned the code to it's original faulty behaviour, and being illuminated I disabled the auto backlight intensity and changed it to it's lowest setting to enjoy a longer life between battery charges.
TO CREATE YOUR KERNEL:
1) Prepare a working environment
1a) Unpack Sourcery G++. No installation needed, unpacking is sufficient
1b) Unpack the Samsung sources and cd to the location of the top-level Makefile.
1c) Unpack the zImage and the contents of the Tweak-Kit to the same location
1d) Make sure the zImage is called zImage.I9000XWJP6
2) The ramdisk image is required and can be extracted from zImage.I9000XWJP6
2a) Create an uncompressed image Image.I9000XWJP6
Code:
gcc scripts/binoffset.c -o scripts/binoffset
ofs=`scripts/binoffset zImage.I9000XWJP6 0x1f 0x8b 0x08 0x00 2>/dev/null`
dd ibs=$ofs skip=1 <zImage.I9000XWJP6 | gzip -c -d >Image.I9000XWJP6
2b) The Tweak-Kit includes mkvmlinux which converts the uncompressed binary image into a bfd object. You need an Arm enabled libbfd to get it working. This does not get installed by default so you need to deeplink into binutils. mkvmlinux locates and decodes the kallsym data and econstructs the symbol table. It then uses the values of __initramfs_start/end to extract the initramfs. If you are not bothered with the hassle, just use dd with hardcoded values.
Code:
g++ mkvmlinux.cc -o mkvmlinux -lbfd -liberty -lz [-I and -L that deeplink into binutils]
./mkvmlinux Image.I9000XWJP6 vmlinux.I9000XWJP6 -r initramfs.cpio
or
Code:
dd if=Image.I9000XWJP6 of=initramfs.cpio bs=1 count=2739712 skip=165568
3) Patch date/time and other environmental issues to the moment of original creation
Code:
patch -p1 <stamp.patch
4) Make your computer happy
Code:
# edit Makefile line 184 and update the macro CROSS_COMPILE=
cp I9000XWJP6_defconfig arch/arm/configs/I9000XWJP6_defconfig
make I9000XWJP6_defconfig
make
5) Verify that the kernel is identical
Code:
diff zImage.I9000XWJP6 arch/arm/boot/zImage
AND NOW FOR SOMETHING COMPLETELY DIFFERENT...
Tweaking the configuration will build you a new kernel but when your Galaxy powers on it will either die silently (hang) or experience a horrible death (reboot). The problem is that there are modules built from sources located in the removed directories drivers/fsr and fs/rfs. These modules were compiled with a specific data structure layout and entry points. These will surely change when re-configuring. The way to keep the non-native modules happy is to keep the structures and entry points intact.
The structure layout is influenced by the CONFIG_ macros. The structures can be fixed to reflect the state of the I9000XWJP6 kernel by replacing the CONFIG_ macro's by something that does not change after reconfiguration. For that I use a collection of 'shadow' macro's which have SHADOW_ as prefix. Because the data structures cannot expand, you cannot (easily) enable configure functionality which require extra fields in the data structures. Reducing functionality is highly seldom a problem.
If changing kernel functionality results in removal of entry points, then stubs are required for those entry points needed by the non-native modules
There are automated methods to verify that a new kernel abides to the above constraints. For the data structures the compiler must generate gstabs debug information. This is human readable and includes detailed structure descriptions. This information should be identical across re-configuration. However, the scripts get confused by anonymous structs which are by product of "typedef struct {" constructions. These need to be named, something shadow.patch also does.
The kernel modules have easily-readable symbol tables containing needed kernel entry points. These should all be present in all re-configured kernels. Validation tests that fail emit enough information to further fix data structures and entry points. The Tweak-Kit contains two files: I9000XWJP6.h containing the SHADOW_ macro's and I9000XWJP6.c for the stubs. Both were constructed in an on-demand basis for the reconfiguration I am currently using and both serve as good examples on what to do when validation fails.
Before reconfiguring, rebuild the kernel for usage as a validation checkpoint.
1) Undo the timestamp patches
Code:
patch -R -p1 <stamp.patch
2) Fix the warnings
Code:
patch -p1 <style.patch # style related issues
patch -p1 <prototype.patch # prototype related issues
patch -p1 <error.patch # bug fixing
3) Apply datastructure fixation, entrypoint stubbing and Makefile tweaking
Code:
patch -p1 <shadow.patch
cp I9000XWJP6.c arch/arm/plat-samsung
4) Before recompiling everything, you need to issue "make clean" first. However, the missing directories will now pose a problem as "make clean" will include their Makefiles and will fail if it can't. Just create empties to keep the build happy.
Code:
mkdir -p drivers/fsr fs/rfs
touch drivers/fsr/Makefile fs/rfs/Makefile
5) Optionally change the Makefile to point to your favorite compiler/toolchain. Please note that I am using GCC 4.4.5. GCC 4.5.1 is bumping into problems I haven't looked into yet.
Code:
# edit Makefile line 184 and update the macro CROSS_COMPILE
6) This build will generate gstab debug information. Unexpectingly, this might bite when combined with function profiling, so disable that. But do not CONFIG_FUNCTION_TRACER yet as that does more.
Code:
# edit Makefile line 553, disable the line containing KBUILD_CFLAGS += -pg
7) Unpack the initramfs image. The directory /lib/modules needs to be examined/updated
Code:
mkdir initramfs.dir
cd initramfs.dir
cpio -i --make-directories --preserve-modification-time --no-absolute-filenames <../initramfs.cpio
cd ..
8) Repack initfs as a tarball, as make clean will erase all the modules
Code:
tar cf initramfs.tar initramfs.dir
9) The initramfs image will contain new kernel modules. Make sure a new version will get generated.
Code:
# in .config line 80 point to the unpacked initram location
CONFIG_INITRAMFS_SOURCE="initramfs.dir"
# in .config lines 86-89, select your favourite compression
CONFIG_INITRAMFS_COMPRESSION_NONE=N
CONFIG_INITRAMFS_COMPRESSION_GZIP=Y
10) Build a new kernel
Code:
# not cleaning will confuse the verification
make clean
make CONFIG_DEBUG_INFO=y
# install the modules
tar xf initramfs.tar
cp `find drivers -name '*.ko'` initramfs.dir/lib/modules
# rebuild with a fresh new ramdisk image
rm usr/initramfs_data.cpio*
make CONFIG_DEBUG_INFO=y
11) Checkpoint structure/entrypoint information. This is architecture independent.
Code:
# extract structures. They are the entries with :T
objdump -G vmlinux | awk '{ print $7 }' | grep :T | sed 's/([^)]*)/()/g' | sed 's/=\*()//g' | sort -u > gstabs.ckp
# extract the entrypoints
nm vmlinux | grep 'r __ksymtab_' | awk '{ print $3 }' | sort > ksymtab.ckp
12) Do a test-run. Pack zImage and flash with Odin. If your Galaxy is up and running, I strongly suggest you make a backup of your environment. If you later change something and it breaks, then this is the best place to restart.
Code:
cp arch/arm/boot/zImage .
tar cf I9000XWJP6-2.6.32.9-test.tar zImage
13) Make your re-configuration. I really suggest you do not make too many changes in one go because it gives more work when the structure/entrypoint verification fails.
Code:
# re-configure. For this exercise, change the kernel to a more production version
CONFIG_CC_OPTIMIZE_FOR_SIZE=Y
CONFIG_DM_DEBUG=N
CONFIG_S3C_KEYPAD_DEBUG=N
CONFIG_DEBUG_FS=N
CONFIG_DEBUG_KERNEL=N
CONFIG_LATENCYTOP=N
CONFIG_FTRACE=N
CONFIG_ARM_UNWIND=N
CONFIG_DEBUG_USER=N
14) Build a new kernel
Code:
# not cleaning will confuse the verification
make clean
make CONFIG_DEBUG_INFO=y
# install the modules
tar xf initramfs.tar
cp `find drivers -name '*.ko'` initramfs.dir/lib/modules
# rebuild with a fresh new ramdisk image
rm usr/initramfs_data.cpio*
make CONFIG_DEBUG_INFO=y
You get "struct has no member named" errors if you have enabled subsystems that require data structures to change which are incompatible with the non-native modules.
15) Verify structure/entrypoint checkpoint
Code:
# extract/verify structures
objdump -G vmlinux | awk '{ print $7 }' | grep :T | sed 's/([^)]*)/()/g' | sed 's/=\*()//g' | sort -u > gstabs.t
# new/changed structures are tagged with '+'. Display only the changed ones
diff -U0 gstabs.ckp gstabs.t | grep '+' | grep ':T' | sed 's/+//' | sed 's/:T.*/:T/' | while read s; do
grep -q "$s" gstabs.ckp
if [ $? -eq 0 ]; then
echo $s;
fi
done
# extract/verify entrypoints
nm vmlinux | grep 'r __ksymtab_' | awk '{ print $3 }' | sort >ksymtab.t
# extract all entrypoints needed by the modules
nm `find initramfs.dir/ -name '*.ko'` >allkosym
# some symbols are referenced in other modules. Cross-reference and remove from list
grep ' U ' allkosym | sort -u | awk '{ print $2 }' | while read s; do
if ! egrep -q "^[0-9a-f]* . $s\$" allkosym; then
echo $s;
fi
done > allusym
# check that kernel has entrypoints for all final undefined symbols
cat allusym | while read s; do
if ! egrep -q "^__ksymtab_$s\$" ksymtab.t; then
echo $s;
fi
done
Both scripts will generate output if you have enabled subsystems that require datastructures to change. These do not necessarily have to be data structures needed by non-native modules. However, missing entrypoints are those used by the modules. If it's about datastructures, your best chance is to lookup the data type and see if there any #ifdef CONFIG_ macros that need to be changed into #ifdef SHADOW_. If it's a missing entry point, you need to add a stub in I9000XWJP6.c.
16) Do a test-run. Pack zImage and flash with Odin.
Code:
cp arch/arm/boot/zImage .
tar cf I9000XWJP6-2.6.32.9-test.tar zImage
17) If you want more, jump to step 13
18) When you are really done, rebuild a final and fresh kernel and initramfs image with debugging stuff removed. The -gstabs compiler switch slightly influences code generation.
Code:
# not cleaning will confuse the verification
make clean
make
# install the modules
tar xf initramfs.tar
cp `find drivers -name '*.ko'` initramfs.dir/lib/modules
# rebuild with a fresh new ramdisk image
rm usr/initramfs_data.cpio*
make
My uncompressed image has now shrunk from 14700623 to 11822559 bytes.
Happy Hacking...
[...and now to find a better workaround for those non-native modules.]

WoW, Nice work !! very good info for beginners like me

thx a lot for this tut and i've learnt a lot
btw, seems there r some typos or something is missing. i did it with (XXJPO):
Hexabit said:
make I9000XWJP6_defconfig
Click to expand...
Click to collapse
make defconfig I9000XWJP6_defconfig
- modified include/linux/a.out.h by removing the 2nd def for SEGMENT
- changed the boolean to lowercase for .config
PS i use the cpio extracted by myself coz i couldnt enable libbfd on my ubuntu x64

Good tips. Thanks.

Really insightful i hope the dev take all the tweaks into consideration and make a new optimized kernel

good job here!

I think it's the most amazing first post ever! It should be sticked or kept somewhere safe.

Awesome first post. Will have to work through this.

Great post with very interesting findings!
I'm no expert, so maybe my question is a bit silly:
Is Samsung's published code just a buggy and incomplete pre-release debug version? Then how can e.g. Voodoo get a good working kernel?
Or is the official firmware really built of this, so possibly full of strange bugs and missing optimizations?

Related

Google Android and Linux for Kaiser Volume II

The original thread:http://forum.xda-developers.com/showthread.php?t=396782 needs an abridged version.
==================================================
Go to http://www.androidonhtc.com/ if you're just starting as it has the latest info. This thread is to highlight the info from the original thread only.
Latest Builds
Port Status
==================================================
Compiling Android Kernel for Kaiser
Modify initrd.gz files and CPIO handling
system.img mounting, editing and rebuilding with ext2/3
system.img mounting, editing and rebuilding with cramfs
Howto: Pull from git (new/update/resync)
[WIP] Configuring WiFi Interface
==================================================
system.img mounting, editing and rebuilding with ext2/3
seidler2547: Post:
Actually I've played with Android a bit for now, and I changed to ext3. It doesn't only work - it's much faster, too! Startup time during the blinking android is about half of what it was before.
How-To:
Code:
Code:
cd /tmp
# prepare dirs
mkdir a-sys
mkdir a-ext
# prepare image
dd if=/dev/zero of=/where/is/sdcard/system.img.new bs=1M count=64
mkfs.ext3 /where/is/sdcard/system.img.new
# mount old image and copy to new
mount -o loop /where/is/the/system.img a-sys
mount -o loop /where/is/sdcard/system.img.new a-ext
cp -a a-sys/* a-ext/
Now you can unmount the old image and happily edit in the new image. Don't forget to rename the system.img.new to system.img (after you have unmounted it).
In your initrd, in file init, where it says
Code:
losetup /dev/block/loop1 /sdcard/system.img
...
mount -t cramfs -o ro,noatime,nodiratime /dev/block/loop1 /system
change "-t cramfs" to "-t ext2" or "-t ext3". You can also change the path (/sdcard/system.img) there.
Click to expand...
Click to collapse
Modify initrd.gz files and CPIO handling
dcordes: Post:
There is no magick in the initrd files. They are .cpio.gz files, gzipped cpio balls. To extract a .cpio.gz file named initrd-android.cpio.gz simply do
Code:
gunzip initrd-android.cpio.gz && cpio -i < initrd.android.cpio
Then you have the extracted rootfs. The reverse way would be, assuming you are inside your rootfs folder:
Code:
find ./ | cpio -H newc -o | gzip > ../my-initr-android-with-custom-stuffs.cpio.gz
And yes, you can remove and add applications you find that way.
Click to expand...
Click to collapse
system.img mounting, editing and rebuilding with cramfs
dzo: Post:
Hi, you can't just use mkcramfs on the system folder because the permissions will be wrong. This is the script I use:
Code:
Code:
out/host/linux-x86/bin/genext2fs -d out/target/product/generic/system -b 80000 -a system.ext2
mount -o loop system.ext2 /mnt/system
cp /mnt/system/usr/keychars/qwerty2.kcm.bin /mnt/system/usr/keychars/vogue-ts.kcm.bin
cp com.google.android.maps.jar /mnt/system/framework
cp Maps.apk Street.apk /mnt/system/app
mkfs.cramfs /mnt/system system.img
umount /mnt/system
#pcp system.img :/Storage\ Card/system.img
This also puts the maps app in (just copy from one of my images) and the vogue keymap. Without the source for the ril you will also need to copy my RIL (libreference-ril.so).
Click to expand...
Click to collapse
[WIP] Configuring WiFi Interface
This has been able to initialize the interface, assign arbitrary IP addresses but can not go further at the moment.
Code:
# ifconfig tiwlan0 192.168.1.100
# ifconfig tiwlan0 up
error: SIOCSIFFLAGS (Cannot assign requested address)
# ifconfig tiwlan0
tiwlan0: ip 192.168.1.100 mask 255.255.255.0 flags (down broadcast multicast)
dmesg will show:
Code:
wlan: no version for "struct_module" found: kernel tainted.
TIWLAN: Driver loading
trout_wifi_power: 1
trout_wifi_reset: 0
trout_wifi_set_carddetect: 1
TIWLAN: Found SDIO control (vendor 0x104c, device 0x9066)
TIWLAN: Driver initialized (rc 0)
TIWLAN: Driver loaded
Android's built-in wireless settings seem to disable the interface beyond just interfering with it, therefore it's best to stay with terminal and using 'ash' will give you a shell with command history (up/down scroll).
markya23: Post:
Need to create a folder in you system image package called /etc/wifi and copy tiwlan.ini, wpa_supplicant.conf and fw1251r1c.bin.
Need to copy the wlan.ko to /lib/modules in the system image (create the dir if required).
Create the new system image and boot Android. Start the dev console and type:
Code:
cp /system/etc/wifi/wpa_supplicant.conf /data/misc/wifi/wpa_supplicant.conf
insmod /system/lib/modules/wlan.ko
wlan_loader -f /system/etc/wifi/Fw1251r1c.bin -e /proc/calibration -i /system/etc/wifi/tiwlan.ini
cd /data/local/tmp
wpa_supplicant -f -Dtiwlan0 -itiwlan0 -c/data/misc/wifi/wpa_supplicant.conf &
ifconfig tiwlan0 192.168.1.100 netmask 255.255.255.0
ifconfig tiwlan0 up
Click to expand...
Click to collapse
Compiling Android Kernel for Kaiser
dwaradzyn: Post:
Here are brief instructions on how to compile android kernel for Kaiser from git.linuxtogo.org repository. I assume that running OS is Linux and it has everything required to build x86 or ia64 kernel. Beside that latest git software should be installed. The shell is assumed to be bash.
1. Let's start with creating a directory for kernel in home directory:
Code:
mkdir ~/android-kernel
cd android-kernel
2. Next thing is to get the sources from repository. To make it happen (this could take a while, it downloads 280MB):
Code:
git clone git://git.linuxtogo.org/home/groups/mobile-linux/kernel.git
OUTPUT:
Code:
Initialized empty Git repository in /home/user/android-kernel/kernel/.git/
remote: Counting objects: 908251, done.
remote: Compressing objects: 100% (153970/153970), done.
remote: Total 908251 (delta 755115), reused 906063 (delta 753016)
Receiving objects: 100% (908251/908251), 281.86 MiB | 292 KiB/s, done.
Resolving deltas: 100% (755115/755115), done.
Checking out files: 100% (22584/22584), done.
3. The htc-msm branch is of our interest (again it could take a few seconds):
Click to expand...
Click to collapse
*** Update, poly_poly-man states we are working off of htc-vogue not htc-msm. I'm leaving the original code here but I would urge you to modify the next line as poly has suggested:
Code:
cd kernel
git checkout -b htc-msm origin/htc-msm
OUTPUT:
Code:
Branch htc-msm set up to track remote branch refs/remotes/origin/htc-msm.
Switched to a new branch "htc-msm"
4. Let's take care of arm toolchain. Download this file (64MB) into ~/android-kernel:
Code:
[url]http://www.codesourcery.com/gnu_toolchains/arm/portal/package2549/public/arm-none-linux-gnueabi/arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2[/url]
Unpack it:
Code:
cd ~/android-kernel
tar xjf arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
5. Compile the kernel
Prepare default .config for Kaiser:
Code:
cd ~/android-kernel/kernel
make htckaiser_defconfig ARCH=arm
OUTPUT:
Code:
........
lots of output
........
# configuration written to .config
#
And finally compile the kernel to get zImage (takes a minute or two):
Code:
export PATH=~/android-kernel/arm-2008q1/bin:$PATH
make zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
OUTPUT:
Code:
........
lots of output
........
Kernel: arch/arm/boot/zImage is ready
Now copy ~/android-kernel/kernel/arch/arm/boot/zImage to your phone and play with it.
Some ending tips:
A. You can compile earlier versions of sources in repository. To do that click on one of "commit" links on page:
Code:
http://git.linuxtogo.org/?p=groups/mobile-linux/kernel.git;a=summary
and read commit id (for example: f9d1bcea9342348623f5a57588044f76d8b649cd):
Code:
git reset --hard f9d1bcea9342348623f5a57588044f76d8b649cd
It will override any changes you made to files in ~/android-kernel/kernel.
B. Once you have downloaded git repository, you can swallow latest changes by issuing:
Code:
cd ~/android-kernel/kernel
git pull
C. If your machine has more than one cpus/cores you can speed up kernel compilation by adding -j <cores/cpus_number>, for example (dual core):
Code:
make -j 2 zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
Click to expand...
Click to collapse
bad internet makes for double post. Please delete
wrong directions for kernel - we are working off of htc-vogue branch... not htc-msm...
can we make the internet work through the usb cable?
how does android know which device node is gps? it's not picking it up for kaiser...
if you enable gps in winmo (or enable it in smd0 - I believe the command is @startgps), smd7 is a nmea stream (acts as a serial GPS).... could a symlink possibly be the right solution to this?
Howto: Pull from git (new/update/resync)
This will download the latest from git:
dwaradzyn: Post:
Here are brief instructions on how to compile android kernel for Kaiser from git.linuxtogo.org repository. I assume that running OS is Linux and it has everything required to build x86 or ia64 kernel. Beside that latest git software should be installed. The shell is assumed to be bash.
1. Let's start with creating a directory for kernel in home directory:
Code:
mkdir ~/android-kernel
cd android-kernel
2. Next thing is to get the sources from repository. To make it happen (this could take a while, it downloads 280MB):
Code:
git clone git://git.linuxtogo.org/home/grou
ps/mobile-linux/kernel.git
OUTPUT:
Code:
Initialized empty Git repository in /home/user/android-kernel/kernel/.git/
remote: Counting objects: 908251, done.
remote: Compressing objects: 100% (153970/153970), done.
remote: Total 908251 (delta 755115), reused 906063 (delta 753016)
Receiving objects: 100% (908251/908251), 281.86 MiB | 292 KiB/s, done.
Resolving deltas: 100% (755115/755115), done.
Checking out files: 100% (22584/22584), done.
3. The htc-msm branch is of our interest (again it could take a few seconds):
Code:
cd kernel
git checkout -b htc-msm origin/htc-msm
OUTPUT:
Code:
Branch htc-msm set up to track remote branch refs/remotes/origin/htc-msm.
Switched to a new branch "htc-msm"
4. Let's take care of arm toolchain. Download this file (64MB) into ~/android-kernel:
Code:
http://www.codesourcery.com/gnu_toolchains/arm/portal/package2549/public/arm-none-linux-gnueabi/arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
Unpack it:
Code:
cd ~/android-kernel
tar xjf arm-2008q1-126-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
5. Compile the kernel
Prepare default .config for Kaiser:
Code:
cd ~/android-kernel/kernel
make htckaiser_defconfig ARCH=arm
OUTPUT:
Code:
........
lots of output
........
# configuration written to .config
#
And finally compile the kernel to get zImage (takes a minute or two):
Code:
export PATH=~/android-kernel/arm-2008q1/bin:$PATH
make zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
OUTPUT:
Code:
........
lots of output
........
Kernel: arch/arm/boot/zImage is ready
Now copy ~/android-kernel/kernel/arch/arm/boot/zImage to your phone and play with it.
Some ending tips:
A. You can compile earlier versions of sources in repository. To do that click on one of "commit" links on page:
http://git.linuxtogo.org/?p=groups/mobile-linux/kernel.git;a=summary
and read commit id (for example: f9d1bcea9342348623f5a57588044f76d8b649cd):
Code:
git reset --hard f9d1bcea9342348623f5a57588044f76d8b649cd
It will override any changes you made to files in ~/android-kernel/kernel.
B. Once you have downloaded git repository, you can swallow latest changes by issuing:
Code:
cd ~/android-kernel/kernel
git pull
C. If your machine has more than one cpus/cores you can speed up kernel compilation by adding -j <cores/cpus_number>, for example (dual core):
Code:
make -j 2 zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
Click to expand...
Click to collapse
poly_poly-man: Post:
This will pull from git which will update/resync a git pull:
Code:
mkdir foo
cd foo
git init
git pull git://git.linuxtogo.org/home/groups/mobile-linux/kernel.git htc-vogue
Click to expand...
Click to collapse
I'll follow-up with some nice, full instructions...
1. prerequisites: arm-none-linux-gnueabi toolchain (gentoo users can use crossdev - otherwise.... uhh... idk?), git, a host toolchain (gentoo users have this by default, most other distros have this under "development" in their package managment... if you have gcc, you're probably set).
2. mkdir kernel
cd kernel
git init
git pull git://git.linuxtogo.org/home/groups/mobile-linux/kernel.git htc-vogue
3. make vogue_defconfig
4. make
5. cp arch/arm/boot/bzImage /path/to/sdcard/
6. to update, run the git pull command by itself again, run make (may have to do the config line again if it's changed) and cp.
Thanks for the post poly. I don't know how that's different from what the post I referenced as I'm not +4 at this stuff.. I did what you posted with android kernel from git and got a 1.2mb zImage that crashed HaRET.. I'm guessing this is my bad. What could I have overlooked? Thanks
enatefox said:
Thanks for the post poly. I don't know how that's different from what the post I referenced as I'm not +4 at this stuff.. I did what you posted with android kernel from git and got a 1.2mb zImage that crashed HaRET.. I'm guessing this is my bad. What could I have overlooked? Thanks
Click to expand...
Click to collapse
where'd you get your toolchain?
does building a regular (host arch) kernel work?
Also - what's the proper way to build a system.img by hand? I'm looking to modify that quite a bit, but can't find a persistent source tree besides the main one, which is seriously crippled.
I thought you were one of the experts, lol. I've been left with no support on how dzo, et all are customizing kernels so I've been in read only mode on the 'other thread' looking elsewhere for support.
Just wanted to ask you first, what's with the Dream radio? I know you posted the mods censored it but what's with the sig now? It piqued my interest... as I'm using (shudder) winmo on the regular while Android is being worked on I was hoping it would be worth looking into if it doesn't brick my phone.
Answers to your questions:
As I said, my own zImage is no go. Check this link (not for our phone but the links at the bottom are pretty useful): http://wiki.xda-developers.com/index.php?pagename=BlackstoneLinux#Runningx20.Linuxx20.onx20.blackstone
I got the toolchain from the steps I (re)posted on this thread:
http://forum.xda-developers.com/showpost.php?p=2269384&postcount=184 so that gave me a 1.2mb zImage where everyone's been posting 1.4mb-- I know there's something not right. As far as building a system.img by hand? I've taken existing ones either from posted bundles or from Android src directly. Maybe I suck (real possibility) but cupcake and 1.0 have been pretty flaky for me (there are system.img's included in the source). You should know how to mount and edit them though (look at the first post on this thread). My experience is the git source is useless unless you've got a G1-- I don't know how to make it run on Tilts. If it does work, then the answer to your question about host arch compiling is no-- it has to be ARMv5 for our phones. That's where this line comes in:
make zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
Click to expand...
Click to collapse
Seriously, I need help learning how to modify the kernel-- I've been a "google will have the answer for me" Linux bystander for a while and while I'm good at what I've done I'm not so good at this uncharted territory. I'm really looking for help to figure out how to compile modules (saurik and dzo never got back to me about that) and all I've gotten is "wait while I do it myself" which is cool they're working on it but we're obviously here to work on it too.
I've even been trying to get Debian installed (familiar territory for me) so I can at least get stuff working. You do know that Android is nothing but a Java VM layer for Linux and it will (could be) put on any self-respecting distro. Personally, I want Debian on my phone with an Android chroot as that would trump all.
Sorry to rant but you seem more about figuring this out like me and I don't know who else is really working on this besides the dev-gods who have no tutorials.
enatefox said:
I thought you were one of the experts, lol. I've been left with no support on how dzo, et all are customizing kernels so I've been in read only mode on the 'other thread' looking elsewhere for support.
Just wanted to ask you first, what's with the Dream radio? I know you posted the mods censored it but what's with the sig now? It piqued my interest... as I'm using (shudder) winmo on the regular while Android is being worked on I was hoping it would be worth looking into if it doesn't brick my phone.
Click to expand...
Click to collapse
It never actually worked... maybe. My phone was reporting the wrong version on a *different* radio (1.65.21.18, was saying 19) before, and trying to flash this changed the version to be correct. 0x300 radios will never flash, and this as a 0x301 *will* brick your phone. Then again... like 2 people reported epic success... In other words, no, it never really existed.
Answers to your questions:
As I said, my own zImage is no go. Check this link (not for our phone but the links at the bottom are pretty useful): http://wiki.xda-developers.com/index.php?pagename=BlackstoneLinux#Runningx20.Linuxx20.onx20.blackstone
I got the toolchain from the steps I (re)posted on this thread:
http://forum.xda-developers.com/showpost.php?p=2269384&postcount=184 so that gave me a 1.2mb zImage where everyone's been posting 1.4mb-- I know there's something not right. As far as building a system.img by hand? I've taken existing ones either from posted bundles or from Android src directly. Maybe I suck (real possibility) but cupcake and 1.0 have been pretty flaky for me (there are system.img's included in the source). You should know how to mount and edit them though (look at the first post on this thread). My experience is the git source is useless unless you've got a G1-- I don't know how to make it run on Tilts. If it does work, then the answer to your question about host arch compiling is no-- it has to be ARMv5 for our phones. That's where this line comes in:
make zImage ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-
Click to expand...
Click to collapse
unnecessary - zImage is implied, and the other two are in the Makefile already.
Seriously, I need help learning how to modify the kernel-- I've been a "google will have the answer for me" Linux bystander for a while and while I'm good at what I've done I'm not so good at this uncharted territory. I'm really looking for help to figure out how to compile modules (saurik and dzo never got back to me about that) and all I've gotten is "wait while I do it myself" which is cool they're working on it but we're obviously here to work on it too.
Click to expand...
Click to collapse
we have 0 modules at the moment (but perhaps support - I forget). Just compile stuff in - modules are very bad.
I've even been trying to get Debian installed (familiar territory for me) so I can at least get stuff working. You do know that Android is nothing but a Java VM layer for Linux and it will (could be) put on any self-respecting distro. Personally, I want Debian on my phone with an Android chroot as that would trump all.
Click to expand...
Click to collapse
It's a nice idea, but remember where a lot of the current coding is taking place - the ril, which is part of android (the system.img, at least), and doesn't go across to other distros quite as well. I think running Dalvik alongside whatever you're running in Debian might be too much for this phone's epically slow processor (compared to msm7201a).
Sorry to rant but you seem more about figuring this out like me and I don't know who else is really working on this besides the dev-gods who have no tutorials.
Click to expand...
Click to collapse
my instructions should work - they are what I use, at least.
Someone should change the instructions to (in the Wiki they are correct):
make ARCH=arm vogue_defconfig
I compiled my kernel, booted in Ubuntu, but had no touchscreen at all, I am not sure if the vogue_defconfig file takes care of everything necessary, so now I am checking everything via menuconfig to see whether eveything is all right or not.
DOMy
Do not Use Ext3 on SD cards
enatefox said:
seidler2547: Post:
Click to expand...
Click to collapse
You should NOT be using ext3 on any sort of flash ram device. You will wear out the medium. Journaling is just a bad idea in this sort of situation.
http://www.handhelds.org/hypermail/familiar/273/27320.html
3) ext3 is "very bad" because of the way it does journaling. It does journal, which seems like a good idea, but it also automatically periodically writes a lot of things out to the same secors on disk. I don't have first hand experience with this, but I remember somebody familiar with ext3 writing about this. It's method of journaling is not particularly intended for any kind of wear leveling at all.
Click to expand...
Click to collapse
http://www.mail-archive.com/[email protected]/msg38988.html
There are three disadvantages with the journaled file system:
- lower performance at write time, since there is the extra work of the
journal
- increased chance of damaging the SD card due to extra use of the
journal causing wearing
- increased space usage (for the journal)
Click to expand...
Click to collapse
And this is the general consensus within most of linux on ext3 and wearing mediums. I'm not sure if Andriod's kernel can do ext4, but ext4 can run without a journal.
Yet another reason not to use ext3 is that is near impossible to undelete something, something you can do with ext2.
-edit-
It looks like Android can use Ext4
http://thatsbadass.com/android/tag/ext4/
haha! is a good job, i like it
can't run android on my kaiser
hi folks,
i have read many threads here and on androidonhtc.com, i have read also the install instructions, but it seems im too stupid to run it.
it fails on "can't find system.img". i wil not flash it, i will run it from sd-card.
so, please can anyone attached an actually zip file which i just unpack und run haret.exe to work android on my kaiser ?
thanks and best regards
lenzen

[APP][27-Dec-09] LuaTool 1.2 - Lua Decompiler, Compiler and Compare

Intro:
This is an all-in-one tool for decompiling, compiling and comparing lua scripts found in Manila (TouchFLO 3D / Sense).
All this is a continuation of sztupy's original work: Lua 5.1 tools.
General:
LuaTool consists of 4 parts: Lua decompiler, Lua compiler, Lua compare utility and a Manila file type detection utility.
LuaDec 3.2 - Lua decompiler
Notes on latest version:
Major overhaul of the local finding algorithm. Most lua scripts can now be fully decompiled without a problem.
Manila 2.5.1921 has a total of 703 scripts (including embedded scripts). LuaDec can fully decompile 663 files. That's a success rate of 94.31%.
General notes:
LuaDec automatically checks if the output file was decompiled successfully.
If it wasn't, LuaDec will also output the disassembly and compare file.
In case the decompile was 100% good, LuaDec will only output the standard .lua file as before.
LuaC 1.2 - Lua compiler
Binary function replacement:
LuaC can directly replace functions in compiled luac files. This can be useful if the luac file can't be fully decompiled, but only a small part of the file needs to be edited. Some more info on function replacement.
Continue statement:
The "continue" statement has been added to the Lua Compiler.
Lua doesn't officially support continue statements, but it looks like HTC added it for their needs, so I'm following their lead.
Usage and versions:
Code:
LuaTool 1.2 by Co0kieMonster
Usage: LuaTool <task_select> [task_options] <task_input>
Tasks:
/decompile (or /d) -- Lua Decompiler
/compile (or /c) -- Lua Compiler
/compare (or /cr) -- Lua Compare utility
/detect (or /dt) -- Manila file type detect utility
LuaDec 3.2
Usage: LuaTool /decompile [options] <inputfile>
Available Options:
-o <filename> specify output file name
-dis don't decompile, just disassemble
-f <number> decompile/disassemble only function number (0 = global block)
LuaC 1.2
Usage: LuaTool /compile [options] <inputfile>
Available Options:
-o <filename> specify output file name
-s strip debug information
-r <n> <luac_file> replace function <n> in <luac_file> with <inputfile>
LuaCompare 1.2.1
Usage: LuaTool /compare [options] <original.luac> <newfile.lua(c)>
Available Options:
-o <filename> specify output file name
-s side by side file comparison
-du disable underline
ManilaDetect
Usage: LuaTool /detect <inputfile>
LuaTool changelog:
# LuaTool v1.2
-updated LuaDec to v3.2, LuaC to v1.2 and LuaCompare to v1.2.1
# LuaTool v1.1
-updated LuaDec to v3.1, LuaC to v1.1 and LuaCompare to v1.2
LuaDec changelog:
# LuaDec v3.2
-Local guesser improvements
---major overhaul - gives much better results
-Conditionals handling improvements
---fixed elseif not being recognised in some cases
---added partial support for complex inline boolean assingment
-General improvements
---fixed single function decompile
---fixed table assignments where there are more then 16 values
---better error handling
# LuaDec v3.1
-Conditionals handling improvements
---wrote a brand new algorithm for handling complex logic expressions
---fixed falsely detected generic for loops
---fixed misplaced if end, because of end-to-break optimization
-Local guesser improvements
---declarations at CALL before RETURN
-General improvements
---fixed indents not behaving properly in some cases
---fixed LOADNIL assignments where the destinations are local variables
---decompiler now displays success rate after decompile
---added SETLIST handling
# LuaDec v3.0.4
-General improvements:
---added back error messages
---fixed variable arguments handling
---fixed multiple inline assignments
---fixed a rare if ending misplacement
-Local guesser improvements at:
---inline bool assignments
---table in table situations
---TAILCALLs
---CALLs which return multiple results
---locals declared just before TEST ops
---SETTABLE where b isn't a constant
# LuaDec v3.0
-core rewrite and cleanup
-more accurate especially with conditionals and loops
-some miscellaneous accuracy improvements
-added extra info to script header (date, time, file name and manila name)
-LuaCompare updated to v1.0.1 (compatibility)
# LuaDec v2.1
- Less crashing:
--- added a failsafe for crashing on bad registers
--- fixed crash on SETUPVAL
--- fixed crash on SETLIST
- Better conditional handling:
--- fixed handling of deeper nested else and elseif
--- fixed handling of empty if-end and else-end blocks
--- added break handling
- Better table handling:
--- fixed inline table assignments
--- fixed handling of numerically indexed tables
- Adjustments to local guesser:
--- fixed guessing for inline table assignments
--- fixed guessing for SETGLOBAL and SETUPVAL at PC 1
LuaC changelog:
# LuaC v1.2
-added binary function replacement
# LuaC v1.1
-added "continue" statement
LuaCompare changelog:
# LuaCompare v1.2.1
-small change to support single function decompile
# LuaCompare v1.2
-pre-compare disassembly is now done internally instead of writing to disk and reading
-added a console message with match percentage
# LuaCompare v1.1
-initial version integrated in LuaTool
Go co0kiemonster! You da man!
boy oh boy ... cant believe that, thanks
time to get back to the keyboard and do some hack0r's stuff
see you guys
I like the new compare output a lot! Saves some lines in the manilatool.cmd as well. Do you plan on updating all the ruby tools or just the compare?
Muchos gracias
12aon said:
Do you plan on updating all the ruby tools or just the compare?
Click to expand...
Click to collapse
Probably all (except luadecguess, which is redundant because luadec has an internal guesser since version 2.0). But I hadn't planned on doing it any time soon - right now, luadec is keeping me pretty busy. I'm doing a semi-rewrite of it in order to inject some OOP love (port to C++) and then hopefully make a proper conditionals and loops engine.
I don't mind OOP love . Hey I somebody came with this idea about luadec but as it turned out I misunderstood him. He was actually talking about the m9editor. Nevertheless the idea is good. You tell me if it's doable.
Wouldn't it be a good idea to include the full manila name in the lines of code as well (If known). Going a bit further might it not be an even better idea to include some more diagnostic info there.
Thing I can think of are manila version (although I can't imagine a foolproof method), date, full manila path name maybe some diagnostics.
You know I'm going to keep you occupied right?
12aon said:
Wouldn't it be a good idea to include the full manila name in the lines of code as well (If known). Going a bit further might it not be an even better idea to include some more diagnostic info there.
Thing I can think of are manila version (although I can't imagine a foolproof method), date, full manila path name maybe some diagnostics.
Click to expand...
Click to collapse
Full manila name and date aren't a problem. I'll add them in the next release.
Manila version would have to be set by the user so that's a bit problematic. But it would be great to have. I'll try to think of good way to add it.
As for diagnostics: Did you mean adding something other than the "-- DECOMPILER ERROR: ... " lines, or just making those lines a bit more useful?
12aon said:
You know I'm going to keep you occupied right?
Click to expand...
Click to collapse
I'm counting on it
Co0kieMonster said:
Full manila name and date aren't a problem. I'll add them in the next release.
Manila version would have to be set by the user so that's a bit problematic. But it would be great to have. I'll try to think of good way to add it.
As for diagnostics: Did you mean adding something other than the "-- DECOMPILER ERROR: ... " lines, or just making those lines a bit more useful?
I'm counting on it
Click to expand...
Click to collapse
The version number can be found in a package here:
Code:
[HKEY_LOCAL_MACHINE\Software\HTC\Manila]
"Version"="2.1.19193517.0"
That's either the .reg or .rgu file
It can also sometimes be found in the package name. But these things are very unpredictable. In that sense it could only be used as an extra. I don't know if any of the exe's in the package hold the info.
By diagnostics I was referring to my lack to come up with anything else. I hoped your developer instincts would lead you to add in the rest for me. But now that I think of it maybe something amount of errors in the script or amount of opcodes, maybe the number of functions. I don't know why, or how it would be useful so probably just leave out that part. Unless you disagree of course,
12
12aon said:
You know I'm going to keep you occupied right?
Click to expand...
Click to collapse
LOL 12 has a new toy!
I guess it would be dumb to ask if you intend to use this in your Manila kitchen! LOL
Asphyx said:
LOL 12 has a new toy!
I guess it would be dumb to ask if you intend to use this in your Manila kitchen! LOL
Click to expand...
Click to collapse
It is already part of the kitchen , co0kie has been helping us for a while now. He is the one who added the lua scheme to notepad2
Ive been trying to use this on the lua files in the sprint hero but no matter what i try i get the error "Bad header in precompiled chunk"
Any thoughts/ideas?
You sure hero's got lua files? Would you mind sharing them?
12
pentace said:
Ive been trying to use this on the lua files in the sprint hero but no matter what i try i get the error "Bad header in precompiled chunk"
Any thoughts/ideas?
Click to expand...
Click to collapse
Might be a different encoding.
Can you upload a few of the files so I can check it out?
Version 3.0 is up
Some info:
Version 3.0 is a complete rewrite of LuaDec. It's more accurate then 2.1, especially when large loops are involved. It might just need a little bit more tweaking but conditional and loop handling is almost perfect. The next big thing to tackle is local guessing, and that will come in a later version.
LuaDec has also generally been cleaned up, so no more obsolete command line switches or memory leaks.
It can also retrieve the full manila name and add it to the file header. E.g.: if you decompile 0bd9db81_manila, LuaDec will add \windows\htc\people\scripts\people\peoplegroupdeta il.luac to the decompiled script header for better reference. For this to work you need to have the m9editor.names.txt file in the same folder as LuaDec.
Now that I've done this rewrite I should be able to accelerate development. And there are some cool new feature coming in future versions.
Decompile Luaplugins for lightroom
Hi,
I just wondering if it is possible to use this to decompile any lua files, the one i'm looking for is decompiling lightroom plugins
skrollster said:
Hi,
I just wondering if it is possible to use this to decompile any lua files, the one i'm looking for is decompiling lightroom plugins
Click to expand...
Click to collapse
LuaDec has been tuned specifically to HTC's Lua variant. Theoretically it should decompile any Lua 5.1 scripts, but it might be incompatible with the character and number encodings of non-HTC scripts. I'm not sure about the specifics, since those adaptation were done before my development efforts - see here for some of the details: http://forum.xda-developers.com/showpost.php?p=3466886&postcount=249
You can always give it a try and see what happens. It can't hurt
Co0kieMonster said:
LuaDec has been tuned specifically to HTC's Lua variant. Theoretically it should decompile any Lua 5.1 scripts, but it might be incompatible with the character and number encodings of non-HTC scripts. I'm not sure about the specifics, since those adaptation were done before my development efforts - see here for some of the details: http://forum.xda-developers.com/showpost.php?p=3466886&postcount=249
You can always give it a try and see what happens. It can't hurt
Click to expand...
Click to collapse
It just gave me an almost blank file, the only thing in it was some stuff i guess you add to all files
skrollster said:
It just gave me an almost blank file, the only thing in it was some stuff i guess you add to all files
Click to expand...
Click to collapse
Yeah, that's definitely because of the different encodings. Sorry, but I guess it's not going to work.
Too bad really, is it possible to create a decompiler for the encoding used for adobes applications? if so, is it much work to change it?
I'm not sure. Upload one or two lua files so I can take a look.

[MOD] SHW-M110S Development (with lagfix/root/voodoo sound kernel)

For SHW-M110s development/discussion. (Korean Anycall SGS ONLY yes for any who don't know, we are officially part of the I9000 forum.
Koe1974 suggested this thread and will I think act as a co-OP on this discussion. Look for front page updates, links, whatever from him 3 posts down, (below the lost guy from China) in the future too.
Version 4 released
Version 4.0r1 released to fix Odin flashing problem
link to kernel
with tegrak_voodoo sound (v2) module, tegrak ext4 module, root (superuser.apk) and busybox 1.17.1 optional, safe mount option overides by default, auto detect lag fixed partitions (improves upgradeability and interchageability) [/B][/size] Compatible with previously z4modded ext2 setups. All ROM versions, SK05 through TA13 tested and released (link is below).
안녕하세요 to any of the Korean developers who find this. Please update us in English about what you are doing. Your English is probably MUCH better than my korean, and I live in Korea.. There are some people in China, Iran, Philippines, etc using this device who might be helped.
-------------------------------------------------------------------------------------------------------------------------
Ext4 (tegrak modules) z4build rooted voodoo sounds kernel link
A stock z4moded kernel with added ext4 support ( presently by "stealing" tegrak kernel modules.) and many tweaks to make it actually work.
This is now working with z4control to get an easy to apply ext4 lag fix!
------------------------------------------------------------------------------------------------------------------------------------
Bug discussion here please, usage discussion (how do get the file into odin?) .. maybe the general thread is better.
For now, our rooting guide is linked from my sig also, although the above linked kernel can also be made to provide root with no effort.
This type of initramfs modification can be done by unpacking a stock kernel making customizations and then applying z4build to it. But I used a z4modded kernel and then applied customizations and repacked. I did this because I set up to repack by hand anyway before knowing I wanted to use z4mod and even then, when I thought I needed to pack into a tegrak kernel (which z4build can't do). It turned out to cause complications, but also taught me a good bit about the process and about z4mod, and probably helped me find bugs.
Some other related useful links for Reference:
kernel extraction (commented by me specifically for tegrak lzma compressed initramfs and z4build split initramfs)
http://forum.xda-developers.com/wiki/index.php?title=Extract_initramfs_from_zImage
kernel repacking
http://forum.xda-developers.com/showthread.php?t=789712
It needs the initramfs to be cpio'd already something like this:
Code:
cd $initram
find ./ | cpio -H newc -o > $repackdir/newramfs.cpio
It also needs the editor.sh script modified to point to the cross-compiler.
By default it can only pack you initramfs into an image which previously had an uncompressed initramfs. This can be modified though easily.
kernel compiling
This old thread probably isn't too useful now..
http://forum.xda-developers.com/showthread.php?t=740740&page=2
We have much better info 3 posts down by koe1974.
Kernel sources here:
http://opensource.samsung.com/
search SHW-M110S.
The first froyo update has a nice readme with a link to the compiler (I don't have the link right this moment)
Not sure if we need to figure/find .configs, or if the ones included are ok to start. Just need to compile one once and see.
I got my compiler toolchain here:
http://www.codesourcery.com/sgpp/li...1-188-arm-none-eabi-i686-pc-linux-gnu.tar.bz2
I thought it's the one recommended is the original froyo source from samsung, but Koe says they point to 2009 version, so I don't know now.
............
I flashed the korean voodoo sk22 kernel just to boot it into recovery. It has CW mod recovery. Might be useful. (BTW I don't recommend flashing unfamiliar kernels haphazardly, ex: this one injected a file into my ROM that interefered with z4mod until I realized it ) Mine BTW injects only one empty directory (/etc/init.d) and nothing else. Remove the kernel and all other changes dissappear. (z4control adds a tad more, but very little, most of its additions self destruct after use.)
Post reserved.
Post Removed, as requested.
Sent from my GT-I9000 using XDA App
Links:
Korea's equivalent to XDA
Lilinser's GitHub - kernel repack, deodexer, etc.
Project-Voodoo - initramfs (SK05)
Just for hobbies - Voodoo for SL28
Tegrak Kernel
SHW-M110S intramfs Requires further research.
Older M110S from someone at MIT working on the M110S potential resource
bml7 & initramfs possible resource
more initramfs
initramfs SK05 Tested .. OK
How-Tos:
Basic How-To Build Environment with built kernel test
First this diff is from a z4modded stock SL28 image to the custom kernel image. It's not against stock. So it includes tegrak files and scripts I changed. Also note the -N option. It pretends like files that don't exist do.. Any binary files that "differ" are actually added from tegrak.
Code:
diff -rbpN z4mod_sl28/initramfs/init.rc z4grak-construction-sl28/initramfs/init.rc
*** z4mod_sl28/initramfs/init.rc 2011-01-13 02:20:12.000000000 -0500
--- z4grak-construction-sl28/initramfs/init.rc 2011-01-11 07:43:34.000000000 -0500
*************** loglevel 3
*** 58,63 ****
--- 58,72 ----
mount j4fs /dev/block/stl6 /mnt/.lfs
insmod /lib/modules/param.ko
+ #ext4 modules by woo
+ insmod /tegrak/lib/modules/mbcache.ko
+ insmod /tegrak/lib/modules/jbd2.ko
+ insmod /tegrak/lib/modules/ext4.ko
+
+ # tegrak system lagfix by woo
+ #
+ insmod /tegrak/lib/modules/tegrak_module.ko
+
# Backwards Compat - XXX: Going away in G*
symlink /mnt/sdcard /sdcard
*************** service vt /system/bin/vtserver
*** 728,734 ****
#user system
#group system
-
service dumpstate /system/bin/dumpstate -s
socket dumpstate stream 0660 shell log
disabled
--- 737,742 ----
*************** service dumpstate /system/bin/dumpstate
*** 739,747 ****
# oneshot
-
# Added by z4mod
service z4postinit /init
oneshot
--- 747,761 ----
# oneshot
# Added by z4mod
service z4postinit /init
oneshot
+ #install root ingore the mount type, it doesn't matter
+ # syntax looks a little strange to me.. we'll see if it works
+ mount rfs /dev/block/stl9 /system rw remount
+ cat /sbin/su > /system/bin/su
+ chown root /system/bin/su
+ chmod 4755 /system/bin/su
+ mount rfs /dev/block/stl9 /system ro remount
\ No newline at end of file
diff -rbpN z4mod_sl28/initramfs/lpm.rc z4grak-construction-sl28/initramfs/lpm.rc
*** z4mod_sl28/initramfs/lpm.rc 2011-01-13 02:20:12.000000000 -0500
--- z4grak-construction-sl28/initramfs/lpm.rc 2011-01-11 06:37:28.000000000 -0500
*************** on init
*** 16,21 ****
--- 16,26 ----
insmod /lib/modules/param.ko
insmod /lib/modules/vibrator.ko
+ #ext4 modules by woo
+ insmod /tegrak/lib/modules/mbcache.ko
+ insmod /tegrak/lib/modules/jbd2.ko
+ insmod /tegrak/lib/modules/ext4.ko
+
mount rfs /dev/block/stl9 /system check=no
mount rfs /dev/block/mmcblk0p2 /data nosuid nodev check=no
Binary files z4mod_sl28/initramfs/sbin/sslvpn and z4grak-construction-sl28/initramfs/sbin/sslvpn differ
Binary files z4mod_sl28/initramfs/tegrak/bin/mkfs.ext4 and z4grak-construction-sl28/initramfs/tegrak/bin/mkfs.ext4 differ
Binary files z4mod_sl28/initramfs/tegrak/bin/tune2fs and z4grak-construction-sl28/initramfs/tegrak/bin/tune2fs differ
Binary files z4mod_sl28/initramfs/tegrak/lib/modules/ext4.ko and z4grak-construction-sl28/initramfs/tegrak/lib/modules/ext4.ko differ
Binary files z4mod_sl28/initramfs/tegrak/lib/modules/jbd2.ko and z4grak-construction-sl28/initramfs/tegrak/lib/modules/jbd2.ko differ
Binary files z4mod_sl28/initramfs/tegrak/lib/modules/mbcache.ko and z4grak-construction-sl28/initramfs/tegrak/lib/modules/mbcache.ko differ
Binary files z4mod_sl28/initramfs/tegrak/lib/modules/tegrak_module.ko and z4grak-construction-sl28/initramfs/tegrak/lib/modules/tegrak_module.ko differ
I'm a bit confused about sslvpn It's in my SL28 , it's not in my z4moded SL28. everything else diff as expected. Maybe I just lost it, maybe z4mod removed it. It's a small unimportant mystery.
BTW it looks like what I believe are the recovery keys have changed from SK22 to SL28, so maybe using the wrong kernel breaks something.
and my slightly modified version of the extraction script with commented lines to deal with lzma.
it's much faster (well.. why not), and it handles direcories a little better. It need a "/" somewhere in the file name though so use "./zImage".
Code:
#!/bin/bash
#MUCH faster than dd bs=1 skip=blah
#
# syntas is fastdd file skip <length_in_bytes_optional>
# skip is NOT optional and should be set to 0 read from begining.
#
fastdd () {
#dd with a skip is crazy slower cause it forces bs=1
#credit goes to somebody on the internet.
local bs=1024
local file=$1
local skip=$2
local count=$3
(
dd bs=1 skip=$skip count=0 2>/dev/null
if [[ "$count" != "" ]]; then
dd bs=$bs count=$(($count / $bs))
dd bs=$(($count % $bs)) count=1
else
dd bs=1024 2> /dev/null
fi
) < "$file"
}
zImage=$1
basedir=${1%/*}
echo working directory $basedir
mkdir $basedir/initramfs
outdir=$basedir/initramfs/
#========================================================
# find start of gziped kernel object in the zImage file:
#========================================================
pos=`grep -P -a -b -m 1 --only-matching $'\x1F\x8B\x08' $zImage | cut -f 1 -d :`
echo "-I- Extracting kernel image from $zImage (start = $pos)"
echo
echo "*** Start of compressed kernel image:" $pos
#========================================================================
# the cpio archive might be gzipped too, so two gunzips could be needed:
#========================================================================
fastdd $zImage $pos | gunzip > /tmp/kernel.img
pos=`grep -P -a -b -m 1 --only-matching $'\x1F\x8B\x08' /tmp/kernel.img | cut -f 1 -d :`
#
# Use next one for tegrak secuere 11 SL28
# It's an lzma header
# It's found by looking for 5D 00 in the exact same place as cpio (070701) is found in stock.
# The long string of FF's is the real give away since 5D 00 is too vague.
# pos=`grep -P -a -b -m 1 --only-matching '\x{5D}\x{00}\x..\x{FF}\x{FF}\x{FF}\x{FF}\x{FF}\x{FF}' /tmp/kernel.img| cut -f 1 -d :`
echo
echo "*** gzip position in kernel.img :" $pos "(start of gzipped cpio)"
#===========================================================================
# find start and end of the "cpio" initramfs image inside the kernel object:
# ASCII cpio header starts with '070701'
# The end of the cpio archive is marked with an empty file named TRAILER!!!
#===========================================================================
if [ ! $pos = "" ]; then
echo "-I- Extracting compressed cpio image from kernel image (start = $pos)"
# use either one of the next two lines for gzip
# dd if=/tmp/kernel.img bs=1 skip=$pos | gunzip > /tmp/cpio.img
fastdd /tmp/kernel.img $pos |gunzip > /tmp/cpio.img
# comment above and uncomment one of next two lines for lzma, if decompressing tegrak image.
# dd if=/tmp/kernel.img bs=1 skip=$pos | unlzma > /tmp/cpio.img
# fastdd /tmp/kernel.img $pos | unlzma > /tmp/cpio.img
start=`grep -a -b -m 1 --only-matching '070701' /tmp/cpio.img | head -1 | cut -f 1 -d :`
end=`grep -a -b -m 1 --only-matching 'TRAILER!!!' /tmp/cpio.img | head -1 | cut -f 1 -d :`
inputfile=/tmp/cpio.img
else
echo "-I- Already uncompressed cpio.img, not decompressing"
start=`grep -a -b -m 1 --only-matching '070701' /tmp/kernel.img | head -1 | cut -f 1 -d :`
echo start $start
end=`grep -a -b -m 1 --only-matching 'TRAILER!!!' /tmp/kernel.img | head -1 | cut -f 1 -d :`
echo end $end
inputfile=/tmp/kernel.img
fi
end=$((end + 10))
count=$((end - start))
if (($count < 0)); then
echo "-E- Couldn't match start/end of the initramfs image."
exit
fi
echo "-I- Extracting initramfs image from $inputfile (start = $start, end = $end)"
echo inputfile: $inputfile
echo start $start
echo count $count
echo outdir $outdir
# dd if=$inputfile bs=1 skip=$start count=$count > $outdir/initramfs.cpio
fastdd $inputfile $start $count > $basedir/initramfs.cpio
cd $basedir
basedir=`pwd`
cd $outdir; cpio -v -i --no-absolute-filenames < $basedir/initramfs.cpio
cp /tmp/kernel.img $basedir/
More to come.
I like the way this is going, appagom, please put [MOD] in the title.
GL on the new thread, if you hope to have Koreans stumbled upon the thread it might be good to add more phrases like:
루팅
갤럭시s
안드로이드
프로요
업그레이드
업데이트
I never really fully utilized it myself. I hope you get more done here or I'll just take the move personally. Actually, now that you guys were moving into compiling and building I thought this would come. Also, you should look to get some Soju out of this with some donate links or something, even if you aren't looking to take in any cash perhaps you could use it as a seed pot for bounties.
Most importantly, I need to know what "the lost guy from China" said...ㅋㅋㅋ
@Koe, don't waste your time on Gingerbread, get us Honeycomb
Compile a Kernel in 13 Lucky Steps
If you are not familiar with Linux, you might have a hard time following this. Just as I am writing how to do get setup to compile a kernel and compile Android apps, others have written how to setup VirtualBox, Ubuntu, AndroidSDK, etc. Please see documentation provided by Oracle, Google, Ubuntu, etc. before you ask for help about VirtualBox, Ubuntu and the SDK. Thanks.
This is written for people who have used Linux but have not compiled much. Or for the brave at heart who are looking for a nice weekend project. This will tell/guide you through getting a system setup that will not destroy your existing OS.
My host OS is Ubuntu 10.10 64-bit. I wanted had to make an Ubuntu 10.04 32-bit 64-bit system for development because I didn't want to deal with 64-bit vs. 32-bit issues but since Gingerbread requires a 64-bit compiler, I had to. Might as well just use my host system, but since we're here let's keep going!
So you're about to begin. Let's just get one thing straight. No! This will not result in a kernel you can flash. “Then why do this?, you ask. Ask yourself that!
1. Install VirtualBox and the Oracle VM VirtualBox Extension Pack (Currently 4.0.0 r69151)
http://www.virtualbox.org/wiki/Downloads
2. Download an Ubuntu ISO (I suggest Ubuntu Desktop 10.04 64-bit)
http://www.ubuntu.com/desktop/get-ubuntu/download
3. Create a new VM in VirtualBox (You may/have to modify the settings)
Operating System: Linux
Version: Ubuntu64
Extended Features: IO APIC
Processors: 2
RAM: 2048MB
Video Mem: 128MB
HDD: 32GB
Enable PAE/NX
Shared Drive: (I use a shared folder, more detail later)
Click to expand...
Click to collapse
4. Install VirtualBox Guest Additions
5. Install Ubuntu and Update Ubuntu
TIP: Mount your Shared Folder with fstab (Optional but helpful)
If you chose to use a shared folder you can auto-mount it via /etc/fstab.
NOTE: I use a shared folder named andDEV and I mount it on my desktop (~/Desktop). Below is what I add to my /etc/fstab (You may/have to change it)
Code:
andDev /home/koe/Desktop/andDev vboxsf uid=1000,gid=1000 0 0
Click to expand...
Click to collapse
6. Enable multiverse and partner "Software Sources"
7. Install additional software: NOTE: I would also recommend installing preload, but it is not required.
Code:
sudo apt-get install qt3-dev-tools texinfo git-core gnupg flex bison gperf libsdl-dev libesd0-dev libwxgtk2.6-dev build-essential zip curl libncurses5-dev zlib1g-dev sun-java6-jdk eclipse ia32-libs
8. Get and Setup the AndroidSDK (Everything you need to know is there or just Google for help)
http://developer.android.com/sdk/index.html
NOTE: Ubuntu 10.04 does not have Java 5 in it's repositories. Follow this link to setup to Java 5
9. Get and Setup ADT Plugin for Eclipse (Everything you need to know is there or just Google for help)
http://developer.android.com/sdk/eclipse-adt.html#installing
TIP: At this point you might want to try Google's “Hello, Android” tutorial.
Click to expand...
Click to collapse
10. Download and Install the Sourcery G++ Lite for ARM EABI Toolchain (Currently arm-2010.09)
http://www.codesourcery.com/sgpp/lite/arm/portal/subscription3053
Look for and click the link for Recommended Release
Look for and click the link for IA32 GNU/Linux TAR
Extract the archive. You will have a folder named arm-2010.09
Make a directory in your home directory named CodeSourcery
Ex. mkdir ~/CodeSourcery
copy the entire arm-2010.09 folder into CodeSourcery
Click to expand...
Click to collapse
11. Update your $PATH
You should be familiar with this because you had to do it to setup the AndroidSDK
Append the following to your PATH in .bashrc
Code:
~/CodeSourcery/arm-2010.09/bin
12 Download and Prepare the Samsung Source Code (Currently SHW-M110S_Opensource_Froyo_update2.zip)
http://opensource.samsung.com/
Click Mobile - Mobile Phones
Look for and download SHW-M110S_Opensource_Froyo_update2.zip
Extract the archive. Inside the new folder are two more archives.
Extract SHW-M110S_Kernel.tar.gz Inside there is a new folder Kernel
You can copy this to a more convenient location. I copy it to my desktop.
In the Kernel folder is a file named Makefile. Open it with your editor of choice.
Go to line 184. You will see ...
CROSS_COMPILE ?= /opt/toolchains/arm-2009q3/bin/arm-none-linux-gnueabi-
You have to change it to something like below, but see the koe? That is my username so you have to change it to your username.
CROSS_COMPILE ?= /home/koe/CodeSourcery/arm-2010.09/bin/arm-none-eabi-
Save Makefile.
Click to expand...
Click to collapse
13. Compile a Kernel
NOTES:
1. Do not try to compile the code in your Shared Folder. It will fail.
2. When issuing these commands you will see lots of output during this part, most of which is not useful to you at this point.
3. The amount of time it takes for the final make command to run will depend on your computer.
Click to expand...
Click to collapse
Open a terminal window and move into the Kernel directory. Issue the following commands.
Code:
$ make shw-m110s_defconfig
$ make menuconfig
When the config editor opens do the following:
DOWN ARROW to Userspace binary formats and press ENTER
DOWN ARROW to Kernel support for a.out and ECOFF binaries and press SPACE
RIGHT ARROW to Exit and press ENTER
RIGHT ARROW to Exit and press ENTER
Press ENTER again and it will exit back to the command line
NOTE: If you have a powerful computer and you want to speed up the build time, make can be run as, make -j# The # represents how much it will try to do at once. $ make does 1 operation, make -j3 tries to do 3. I have a 2.66 Ghz dual-core CPU and I allow the VirtualBox guest OS access to both cores, so I use make -j3 The compile finishes in about 12 minutes and allows me to still use my host OS. For now, you might just want to run make without the -j option to get a full understanding of how long it takes. Later you can test with values.
Click to expand...
Click to collapse
WARNING: Time is relative. This will take some time ... go make a sandwich or maybe even watch a movie.
Code:
$ make
When you see $ again check the last couple of lines of output. You want to see …
OBJCOPY arch/arm/boot/zImage
Kernel: arch/arm/boot/zImage is ready
Click to expand...
Click to collapse
Congratulation! You just built a kernel for the m110s!
good write up.. Glad to see you hit the same a.out snag as me. Just makes me think it's not configured right and so I have low hopes. Will be fun to see what happens when you put an initramfs in it. I'd just unpack the stock one and try that first.
As for z4control.. I'm pretty interested in getting this working as something like this was the real reason I started messing with this. It seems there may be some issue with the z4mod's init wrapper not doing things it should but anyway.. just now seeing issues. flashed one kernel where I added some debug output added.. trying to understand it (not understanding it yet). I'm optimistic that I can track it down. edit:... definitely making progress, not quite there yet but getting closer.
appagom said:
good write up.. Glad to see you hit the same a.out snag as me. Just makes me think it's not configured right and so I have low hopes. Will be fun to see what happens when you put an initramfs in it. I'd just unpack the stock one and try that first.
Click to expand...
Click to collapse
Strictly to see if it would build completely and to get more info on how to do it, I did do an initramfs & kernel test build.
used the update2 kernel source
used the initramfs linked above (SHW-M110S intramfs Requires further research.)
ran find ./ | cpio -H newc -o > ~/Desktop/newramfs.cpio
added the cpio via menuconfig with no compression
It did build successfully and I ended up with a 7mb zImage vs. a 2.5mb.
There is no way in hell I am going to flash it because I do not know which initramfs (maybe from sk05) it is or what it contains but it did complete.
Now isn't this a kick in the nuts!
Since I got the toolchain all set I decided to focus on the Android source code. Following these directions .. http://source.android.com/source/download.html I got to "Building the code"
Code:
[email protected]:~/Desktop/myAnd$ source build/envsetup.sh
including device/htc/passion/vendorsetup.sh
including device/samsung/crespo/vendorsetup.sh
[email protected]:~/Desktop/myAnd$ lunch
You're building on Linux
Lunch menu... pick a combo:
1. full-eng
2. full_x86-eng
3. simulator
4. full_passion-userdebug
5. full_crespo-userdebug
Which would you like? [full-eng] 1
============================================
PLATFORM_VERSION_CODENAME=AOSP
PLATFORM_VERSION=AOSP
TARGET_PRODUCT=full
TARGET_BUILD_VARIANT=eng
TARGET_SIMULATOR=false
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv5te
HOST_ARCH=x86
HOST_OS=linux
HOST_BUILD_TYPE=release
BUILD_ID=OPENMASTER
============================================
[email protected]:~/Desktop/myAnd$ make
============================================
PLATFORM_VERSION_CODENAME=AOSP
PLATFORM_VERSION=AOSP
TARGET_PRODUCT=full
TARGET_BUILD_VARIANT=eng
TARGET_SIMULATOR=false
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv5te
HOST_ARCH=x86
HOST_OS=linux
HOST_BUILD_TYPE=release
BUILD_ID=OPENMASTER
============================================
Checking build tools versions...
build/core/main.mk:76: ************************************************************
build/core/main.mk:77: You are attempting to build on a 32-bit system.
build/core/main.mk:78: Only 64-bit build environments are supported beyond froyo/2.2.
build/core/main.mk:79: ************************************************************
build/core/main.mk:80: *** stop. Stop.
Well, 32-bit will do for now seeing how ::cough:: I'm not the one building gingerbread.
I guess my next how-to is going to be how to go out and buy a PC and setup a 64-bit development environment.
Well, 32-bit will do fro now see how ::cough:: I'm not the one building gingerbread.
I guess my next how-to is going to be how to go out and buy a PC and setup a 64-bit development environment.
Click to expand...
Click to collapse
Could try the gnu cross compiler I suppose maybe it's clever enough to work around it. Your computer really isn't 64 bit though? You could just install a 64 bit VM assuming it is.
koe1974 said:
added the cpio via menuconfig with no compression
Click to expand...
Click to collapse
Ok, that procedure is easier than it used to be, or than what I read it used to be. I'm not afraid to add one and flash it. Just busy with making one I like right now though.. then again.. sounds like a 10 minute interruption.. so.. we'll see... oh and I don't remember what was stopping xconfig from working, but something annoying.. it's quite a bit nicer than menuconfig. I need to learn how that works though.. cause really you should add modules from the compilation itself right.. so you cant have the initramfs already before you compile, unless make opens it and add the modules and re-cpio's it. Anyway.. I'll shoot first, ask later.
edit: just flashed.. it gave about 1 tenth of a blue bar and froze. Ok, that was fun, back to fixing and ext4 kernel up.
appagom said:
Could try the gnu cross compiler I suppose maybe it's clever enough to work around it. Your computer really isn't 64 bit though? You could just install a 64 bit VM assuming it is.
Click to expand...
Click to collapse
The VM I setup was 32bit, but it's 64bit now. I will just modify the How-To for a 64bit system.
4 bugs related z4control issues solved(ok 2 were probably the same as well as a fifth, so really 3)..
one, it's failing to create a directory it needs (/system/etc/init.d), reported, work around create by hand.
2) It's rfs formatting wrapper script which checked for voodoo, failed. Strange bug in sh actually.. workaround in script found and reported
bugs 3 and 4 details unimportant, were related to the mystery of where sslvpn went. z4build was splitting the initramfs by tacking it some of it brute force on to the end of the zImage. The init script found it again and unpacked the files before continuing. Oddly, it seems an uneeded because I fit the only big displaced file in just fine without splitting and I didn't even use heavy compression. Anyway, needed or not it caused two files to go missing, this one, and a small text file that reported the version of z4mod. this file prevented z4control from working.
I will not fix this tonight, but I mostly understand it (altough not in exact detail of precisely understanding some of the odd symptoms, but I think dealing with this will likely solve it all) My diffs could never find the version file, cuase I never unpacked it to know it even existed in the first place. Waaaah.. bug tracking is tiring. We should have this all running very shortly.
update: my first attempt at fixing the repacking bug.. resulted (I already feared this but hoped it would just go away) in a kernel that seems totaly complete, but where it seems something in the init wrapper isn't working... getting closer to finding it.
update: LAST BUG FIXED
The last bug(which I previously assumed was part of the repack bug) was actually aslo part of z4build, now documented here:
http://forum.xda-developers.com/showpost.php?p=10638535&postcount=1062
I'm pretty sure that one should really get it all going now. there are no lines of code left to break. I've tested almost every line now. I'll get a new kernel out, but will need to wait for an updated z4control. z4ziggy seems busy right now maybe (no complaint obviously).
I can fix this last bug in my roll out of the kernel. The other remaining ones can be worked around pretty easily, but not pretty for user instructions, better to wait probably.
SK05 Rebuild test (PASS)
We have doubts about if the shw-m110s_defconfig is what is used by samsung so I decided to try to make a working kernel from an initramfs and froyo_update source code.
General idea of what I did...
sk05 source code froyo_update (from samsung)
sk05 initramfs (link in post 4 "initramfs SK05 Tested .. OK")
cd initramfs_dir
find . -print0 | cpio -o0 -H newc | gzip -9 -f > some/path/initramfs.cpio.gz
cd kernel source code root directory
modify Makefile ln. 184
make aries_android_rfs_defconfig
make menuconfig
disable a.out
add path to initramfs archive
compression gzip
make
tar --format=ustar SK05rebuilt.tar zImage
Click to expand...
Click to collapse
Results in a zImage the same size (4.6mb) as samsung's, and it boots.
awesome..
I GOT IT
That was big fat B to solve. Debugging self destructing scripts that run in a startup environment with different mounts and different PATH variable is no fun and requires some imagination. I had to work around 2 more bugs in z4build but now I have user transparent workarounds to all of them.. plug and play. Unfortunately I have about 30 minutes more free time today so I probably wont post it today. I should clean up some comments and such in it first probably. We'll see.
appagom said:
awesome..
I GOT IT
Click to expand...
Click to collapse
Very nice!
Sent from my SHW-M110S using XDA App
the "awesome" comment was referring to you. Should be able to take the grak of z4grak with some compiling , but I'm happy with it in too.. doesn't matter.
Anyway.. I updated the kernel page (from the link on OP). The new kernel is there, but I pushed it out very fast. Didn't flash last repack myself yet after changing comments.. but I only changed comments. (I am running the posted copy now) It needs testing since my system has gotten a been used and non-pristine. Get z4control, you can use it to flash it if you want. the rest is all push button I hope.
Oh and default settings are less safe than Tegrak, not much if any better than ext2 actually. I think.. can't confirm right now. I recommend modifying etc/fstab to data=ordered,barrier=1 personally.. but that's what I like about z4.. you can decide for yourself.
Sent from my SHW-M110S using XDA App

[KERNEL] is Tiny-shmem good for android?

Hi fellows,
I have made a patch to enable the use of Tiny-SHMEM instead the full SHMEM filesystem in android kernels. I think we can gain some performance here. It seems to be working, but I have some thoughts to share with you.
1) It seems that android uses /dev/ashmem rather than /dev/shm, and the only two functions of mm/shmem.c needed by mm/ashmem.c is available on tiny-shmem.
2) Android also needs to mount filesystem as tmpfs. If we use tiny-shmem, than tmpfs will use ramfs code. So, we will support online growing, but we can not limit sizes. I have just one partition (/app-cache) that has size limits. The question is: we actually need size limits?
If anyone like, I can release the patch after some testing I'm doing.
Thanks,
Ronan
I can help u test on, well you know!
Hi diz!
So, I'm attaching a patch for GT-P1000 kernels. It should be fine on every version.
It just modify the Init/Kconfig file since the tiny-shmem is already on source code and you just need to adjust the configuration to define CONFIG_TINY_SHMEM.
If you want to test, apply this patch into your kernel tree, execute 'make menuconfig', go to section:
General Setup -> Configure standard kernel features (for small systems) -> Default shmem filesystem implementation -> Tiny shmem filesystem
and then compile.
Any updates in this project??
Sent from my GT-P1000 using XDA App
jadmask3rlm said:
Any updates in this project??
Sent from my GT-P1000 using XDA App
Click to expand...
Click to collapse
Still need people to test it... RAMFS could induce some problems...
My latest unstable kernel version has this patch applied and tiny-shmem is selected. If anyone wants to test: http://forum.xda-developers.com/showthread.php?t=1274855
But, it is just for LATIN models!
Ronis_BR said:
Hi diz!
So, I'm attaching a patch for GT-P1000 kernels. It should be fine on every version.
It just modify the Init/Kconfig file since the tiny-shmem is already on source code and you just need to adjust the configuration to define CONFIG_TINY_SHMEM.
If you want to test, apply this patch into your kernel tree, execute 'make menuconfig', go to section:
General Setup -> Configure standard kernel features (for small systems) -> Default shmem filesystem implementation -> Tiny shmem filesystem
and then compile.
Click to expand...
Click to collapse
Hello
I have, and then someone else (just to make sure) applied the patch, but the "default shmem filesystem implementation" option is not available in general setup, after running 'make menuconfig'
dizgustipated said:
Hello
I have, and then someone else (just to make sure) applied the patch, but the "default shmem filesystem implementation" option is not available in general setup, after running 'make menuconfig'
Click to expand...
Click to collapse
yea the same happened with me too(on another device too)
so what i did
in .config
add this:
CONFIG_TINY_SHMEM=y (it should be 'is not set')
Hi diz,
This option is under Configure kernel features (for small systems). It is the last option in the menu and you can select two values. Yesterday I applied the patch in my kernel tree and it works properly.
Sent from my GT-P1000L using xda premium
AH! Btw, you must enable:
CONFIG_MMU (System type)
CONFIG_EMBEDDED (General setup)
to see the option. So, if you haven't enabled CONFIG_SWAP, than you can choose CONFIG_TINY_SHMEM, otherwise just CONFIG_SHMEM will be available.
FYI, I'm using this with tiny-shmem for 8h and nothing weird happened yet
Thanks,
Ronan
working on it now,
up to the compile part,
its a little different working with my source, than stock
quick q
how do we show users this is working or even added in their kernels?
Well, you can execute:
busybox zcat /proc/config.gz | grep SHMEM
If you see:
CONFIG_TINY_SHMEM=y
Then tiny shmem was selected and is activated.
I'll wait your results! Thanks for the help!!!!
Ronan
Sent from my GT-P1000L using xda premium
Now, when trying to compile (mine and teks way)
It says, kernel tree not clean, please run make mrproper in kernel directory.
So I do, and it erases the tiny shmem, and goes back to default p1cm7 configuration.
Lol, kernel work makes porting a rom from basecode a walk in the park!
When typing make, or make zimage, in same directory as make menuconfig, it errors and says /opt/toolchains/arm-2009q3/bin/arm-none-linux-gnueabi-gcc: Command not found
and
make
make: /opt/toolchains/arm-2009q3/bin/arm-none-linux-gnueabi-gcc: Command not found
scripts/kconfig/conf -s arch/arm/Kconfig
init/Kconfig:957:warning: choice value used outside its choice group
init/Kconfig:958:warning: defaults for choice values not supported
make: /opt/toolchains/arm-2009q3/bin/arm-none-linux-gnueabi-gcc: Command not found
CHK include/linux/version.h
UPD include/linux/version.h
CHK include/generated/utsrelease.h
UPD include/generated/utsrelease.h
Generating include/generated/mach-types.h
CC kernel/bounds.s
/bin/sh: /opt/toolchains/arm-2009q3/bin/arm-none-linux-gnueabi-gcc: not found
make[1]: *** [kernel/bounds.s] Error 127
make: *** [prepare0] Error 2
below is my .config file, zipped because xda cant load hidden files
Hi diz,
First,
kernel tree not clean, please run make mrproper in kernel directory.
It is because the kernel directory has a git repo and it is not clean. When you use 'make mrproper' it will remove the patch because it probably wasn't commited yet.
The easiest way to solve it is:
mv .git .gitold
<Compile the kernel>
mv .gitold .git
If you do this, this problem should be gone.
The second error seems that you are pointing the gcc cross-compiler to a wrong path in Makefile. Check where you installed the compiler and set it accordingly in Makefile by changing the value of this variable:
CROSS_COMPILE ?=
Hope it helps!
Thanks,
Ronan
By the way, your config file seems to be ok!
Change the tool chain prefix to arm-none-eabi- and compile (after doing the steps ron told)
SUCCESS!!!!!!!!
I've posted it here,
http://forum.xda-developers.com/showpost.php?p=19128294&postcount=2205
Good! Let's see what will happen now
Thanks!
Cyanogen 7 tiny_shmem kernel
I have decided to move the kernel here.
So as not to "invade" another's thread
This is only for unofficial beta cm7 port
* FILE SYSTEM
TINY_SHMEM enabled
* LCD Voltage
down to 260000 from 300000
* I/O scheduler default
"no-op"
type: busybox zcat /proc/config.gz | grep SHMEM
in your terminal emulator
you will see:
CONFIG_TINY_SHMEM=y
(means tiny shmem is activated."
dizgustipated said:
I have decided to move the kernel here.
So as not to "invade" another's thread
* FILE SYSTEM
TINY_SHMEM enabled
* LCD Voltage
down to 260000 from 300000
* I/O scheduler default
"no-op"
type: busybox zcat /proc/config.gz | grep SHMEM
in your terminal emulator
you will see:
CONFIG_TINY_SHMEM=y
(means tiny shmem is activated."
Click to expand...
Click to collapse
THIS KERNEL IS Making my WIFI Hotspot activate on MIUI for SGT
- I just have issue with the Mobile Network being broken?
- The touchscreen also broken but i'ved fixed that one
- usb tethering error = ive'd fixed it too
Can you do something on the mobile data being broken on the kernel side?
Because if i use angel666 zimage , cm modded, mobile data work but hotspot broken
If i use this its the other way around. Lol

[DEV] Building a custom kernel and kernel modules for stock kernel

Since fire phone doesn't have a bootloader unlock at the moment. There is no point in building a custom kernel. But By building a kernel we can build kernel modules which work on the stock kernel. And yes you can load unsigned kernel modules without a problem since fire phone doesn't use tz apps to verify kernel modules like Samsung does.
Setup
Source
Download the fire phone sources for firmware 4.6.1 from here. And extract the platfrom.tar inside the archive to somewhere(KERNEL_DIR).
toolchain
You can use the android ndk from google, But it requires some setup. I'm using linaro toolchain from here. You can use compiler version 4.7, 4.8 or 4.9. Kernel I'm using (Firmware 4.6.3 - Linux 3.4-perf-g280c96c) is built with gcc-4.7. But I'm using this gcc-4.9. Download it, extract is somewhere(TOOLCHAIN_DIR) and add the $TOOLCHAIN_DIR/bin to your PATH. Theoretically you would be able to build the kernel on windows using Cygwin or MSYS tools but using Linux is better.
config
Connect your phone trough adb and run
Code:
adb pull /proc/config.gz
zcat config.gz > $KERNEL_DIR/kernel/qcom/3.4/.config
With this config you will run into some problems because of a missing "trapz_generated_kernel.h". I don't know if this is an auto generated file when they build android as a whole or amazon removed this explicitly(can they do that without violating GPL?). Anyway It looks trapz is some low level kernel debugging function(comment here if you know more about it). We can safely disable it. Open $KERNEL_DIR/kernel/qcom/3.4/.config in a text editor and change the lines
Code:
CONFIG_TRAPZ=y
CONFIG_TRAPZ_TP=y
CONFIG_TRAPZ_TRIGGER=y
CONFIG_HAVOK=y
to
Code:
#CONFIG_TRAPZ=y
#CONFIG_TRAPZ_TP=y
#CONFIG_TRAPZ_TRIGGER=y
#CONFIG_HAVOK=y
building
Now edit the $KERNEL_DIR/kernel/qcom/3.4/Makefile and add this changes
Code:
EXTRAVERSION = -perf-g280c96c
This is at the top of the makefile. If we don't add this, vermagic for the modules will differ from stock kernel and they won't load.
ARCH=arm
CROSS_COMPILE=arm-linux-gnueabihf-
Click to expand...
Click to collapse
Here arm-linux-gnueabihf- is my cross compiler frefix. Look in $TOOLCHAIN_DIR/bin/ to find it.
Now cd into $KERNEL_DIR/kernel/qcom/3.4/ and do
Code:
make
The build will fail a few times complaining about missing headers. Most of the time it's just
Code:
#include <myheader.h>
instead of
Code:
#include "myheader.h"
Edit the source file where the build fails and change <>s to ""s. (maybe android ndk ignores the difference and include the headers anyway)
After kernel compiles, we are good to go. We can use this kernel sources to build kernel modules for stock kernel.
Kernel modules
To build the kernel modules, we basically need two things. An approximate kernel source and the Module.symvers file from the original kernel. We can get the Module.symvers file by building the complete kernel as explained above or Just extract it from our stock kernel.
To extract the Module.symvers from the stock kernel, extract the boot.img file from firmware update image. Get mkbootimg tools from here compile it and run
Code:
unmkbootimg --kernel zImage ---ramdisk ramdisk.cpio.gz -i boot.img
After you get the zImage. Download extract-symvers script from here and run
Code:
python2 extract-symvers.py -B 0xc0008000 zImage > Module.symvers
place this file in $KERNEL_DIR/kernel/qcom/3.4/ (You still have to do the changes mentioned above in kernel config and building section run make in the $KERNEL_DIR/kernel/qcom/3.4 and intrupt it after few seconds)
Now you can build loadable modules against this source. Here is a hello world kernel module.
Code:
//hello.c
#include<linux/module.h>
#include<linux/kernel.h>
#include<linux/init.h>
static int __init hello_start(void)
{
printk("hello to the world from module");
return 0;
}
static void __exit hello_end(void)
{
printk("heloo exit");
}
module_init(hello_start);
module_exit(hello_end);
Code:
#Makefile
KERNEL_DIR=<your kernel dir>/kernel/qcom/3.4
obj-m := hello.o
PWD := $(shell pwd)
default:
$(MAKE) ARCH=arm CROSS_COMPILE=armeb-linux-gnueabi- -C $(KERNEL_DIR) SUBDIRS=$(PWD) modules
Put this files in a folder and run make in it. Change the paths and cross compiler prefix according to your setup. and run make.
After the build push the hello.ko to the phone.
Code:
adb push hello.ko /sdcard/
adb shell
su
cd sdcard
insmod hello.ko
run dmesg and you'll see the message.
I'm currently trying to build kexec module from hashcode's sources and USB OTG modules.
I'm attaching a few thing helped me do this.
since they have released this version of the fire os they have to provide the source code
see
http://www.gnu.org/licenses/gpl-faq.html#GPLRequireSourcePostedPublic
you have just shown that the source code they releases for the kernel does not match the one used to build the kernel. This means it is a clear violation of the gpl and amazon is in breach and can be sued.
on another note.
are the drivers for the nfc and camera compiled as a module or into the kernel?
They have yet to provide 4.6.3 and 4.6.4 kernel sources too.
I don't know exactly but in order for NFC and camera to work drivers are required and they are in fact compiled into the kernel.
The problem we currently have with NFC and camera is proprietary hal (hardware abstraction libraries) They are a part of Android and does not subject to GPL. Amazon changed the original android way how hal works and didn't release the sources!
by looking at the kernel drivers maybe we would be able to implement hal from scratch. But I don't see that intense dev support for fire phone. If you are up for it camera sources are at $KERNEL_DIR/kernel/qcom/3.4/drivers/media/platform/msm/camera_v2/
Major MAJOR respect for all of you making the Fire Phone even better!
@madushan1000
Could we do something like this to install a custom boot.img?
http://forum.xda-developers.com/optimus-l9/general/guide-install-custom-roms-locked-t3249828
I don't own this device but has anyone tried to see if kexec works?
spudowiar said:
I don't own this device but has anyone tried to see if kexec works?
Click to expand...
Click to collapse
Nope, I was working on it. But then I got a job. It will be sometime before I can start working on it again.
Could someone please provide the config extracted from /proc/config.gz?
I can't find this on CM11 rom for some reason.
Building the kernel now.
Some bugs are in the code and -Wall and gcc-wrapper.py escalate the warnings.
I wonder if those errors are there on purpose XD
helloworld.ko loaded successfully
I was able to execute kexec without anything. Just the binary.
Will keep you posted - this hacking might take a while to figure it all out.
I already have 3.4 kernel from the amazon sources.
I have the kexec userland program.
What is left is a loadable kexec kernel module (if that is possible at all).
removed
Okarin said:
Are we even sure those Amazon Kernel Sources are correct?
Those errors caught by the wrapper scripts are giving me the creeps.
Git the kexec_load.ko build.
Currently hands on insmod.
Phone doesn't do a reboot any longer:
insmod kexec_load.ko
init_module(0xb6e6c008, 408241, "") = -1 ENOENT (No such file or directory)
write(2, "insmod: init_module '/sdcard/kex"..., 79insmod: init_module '/sdcard/kexec_load.ko' failed (No such file or directory)
) = 79
munmap(0xb6e6c000, 409600) = 0
mprotect(0xb6f8c000, 4096, PROT_READ|PROT_WRITE) = 0
mprotect(0xb6f8c000, 4096, PROT_READ) = 0
close(0) = 0
close(1) = 0
close(2) = 0
futex(0xb6f6cd74, FUTEX_WAKE_PRIVATE, 2147483647) = 0
munmap(0xb6f8c000, 4096) = 0
exit_group(-1) = ?
First goal is to get module loaded.
Goal reached:
kexec_load 27813 0 - Live 0x00000000 (O)
procfs_rw 12770 0 - Live 0x00000000 (O)
wlan 3793980 0 - Live 0x00000000 (O)
Shouldn't be functional at all..
I disabled some function calls just to get the module loaded.
The missing symbols are:
soft_restart
arch_kexec
machine_shutdown
And the version I use does some insane function hooking ..
More rework is needed.
[email protected]:/data/local # ./kexec /sdcard/vmlinux
kernel: 0xaf12d008 kernel_size: 7e1354c
unrecoverable error: could not scan "/proc/device-tree/": No such file or directory
<6>[ 97.681256] Kexec_load: Replacement... :
<6>[ 97.681344] kexec_load : my_syscall_table : c0106244
<6>[ 97.681405] kexec_load : kexec_load before replacement : c01b346c
<6>[ 97.681480] kexec_load : kexec_load after replacement : bf3a5650
<6>[ 97.681546] kexec_load : reboot before replacement : c01a83f0
<6>[ 97.681616] kexec_load : reboot after replacement : bf3a6348
<6>[ 97.681675] Kexec_load: End replacement... :
<6>[ 202.694691] Kexec: - Starting kexec_load...
<6>[ 202.694849] Kexec: - ---- kexec_load - result : '0'
It gets better:
255|[email protected]:/data/local # ./kexec --dtb=/sdcard/zImage-dtb /sdcard/vmlinux
kernel: 0xaf1b1008 kernel_size: 7e1354c
kexec-zImage-arm : dtb.img BEFORE CUT : Start : '0xae66f008' - Length : '0xb411e9' - End : '0xaf1b01f1'
Segmentation fault
More tomorrow.
Click to expand...
Click to collapse
Where are you getting your kexec module sources from? BTW try using the original amazon kernal binary the phone is shipped with (we are sure it works). Don't use the custom kernel for the kexec tests (We don't know the custom kernel actually works)
madushan1000 said:
Where are you getting your kexec module sources from? BTW try using the original amazon kernal binary the phone is shipped with (we are sure it works). Don't use the custom kernel for the kexec tests (We don't know the custom kernel actually works)
Click to expand...
Click to collapse
Here is the thread I used as a starting point.
I will put up my "fork" on github after I get permission to do that
The userland part build like a charm once I took the compiler you recommended.
The kernel-module was tricky because the whole thing is modded like hell.
To be able to load I had to comment out some hard coded addresses and the calls to unresolvable symbols.
removed
Okay the kernel gets loaded.
But
kexec -e shuts off the device.
strace doesn't help.
On the plus side:
Devices are shutdown
Look promising
I need a way to tail dmesg ...
Okay a lot of digging around and I found out that the reboot syscall doesn't work properly..
It doesn't look like it hits the kexec_module it looks more like it hits the actualy sys_reboot
Okay reboot syscall hits my reboot-hook.
But the softreboot doesn't work now.
Okay there is some kind of watchdog runnig which doesn't like my kexec.
I need to kill it - that should happen tomorrow.
removed
I hit the same wall when I tried to isolate the kexec code from the kernel itself to a module. I stopped working on it because I lacked the time. BTW the error you are facing now
<3>[ 80.580644] BUG: scheduling while atomic: kexec/4067/0x00000002
Click to expand...
Click to collapse
is because memory allocator is trying to switch threads while you are in a syscall. it's because of lines like this
image = kzalloc(sizeof(*image), GFP_KERNEL);
Click to expand...
Click to collapse
Try changing GFP_KERNEL to GFP_ATOMIC. Other than that, I have another suggestion. Try to get the kernel to run in a single core mode before running kexec code. This might simplify things. I don't know how to do this though.
madushan1000 said:
I hit the same wall when I tried to isolate the kexec code from the kernel itself to a module. I stopped working on it because I lacked the time. BTW the error you are facing now
is because memory allocator is trying to switch threads while you are in a syscall. it's because of lines like this
Try changing GFP_KERNEL to GFP_ATOMIC. Other than that, I have another suggestion. Try to get the kernel to run in a single core mode before running kexec code. This might simplify things. I don't know how to do this though.
Click to expand...
Click to collapse
The atmic error is gone now. It went away after I disabled the watchtog.
smp_disable() is what you are looking for - but this causes the system to hard_reboot ATM XD
what happens if you kill every userlevel program before smp_disable()?
removed
#define tomorrow
Okay .. I worked out the preemption thing.
At least it does something.
Still a black screen and the MSM_WATCHDOG is a ***** again.
It needs to be suspended .. at least that what I get from the code I read here.
If I remove the driver too early the output in /proc/kmsg stops ..
If I try to remove it too late ... well it causes a resched while atomic.

Categories

Resources