data2ext for HTC Hero & G2 Touch - Hero, G2 Touch Android Development

Hi!
I was experimenting with the data2ext script made for the Desire in following thread:
http://forum.xda-developers.com/showthread.php?t=859419
Now I managed to get it working with HTC Hero and Flykernel12a and Amon RA-s recovery v1.7.0.1 .
MAKING THIS WORKING IS A RISKY OPERATION! DO NOT START IF YOU DO NOT UNDERSTAND WHAT IS WRITTEN BELOW! I'M NOT RESPONSIBLE IF YOU DAMAGE YOUR PHONE OR YOU LOOSE YOUR DATA!
So here is what I made:
- make backup of your entire SD card, because we are going to repartition it, so all data will be lost on your SD card
- reboot to recovery and MAKE NANDROID BACKUP of your existing configuration!!! Then stay in recovery mode, and copy your NANDROID BACKUP to a safe place on your computer from the SD card!!!!! DURING THE OPERATION WE WILL ERASE THE WHOLE SD CARD!!!
- wipe your cache and dalvik-cache (only this 2, NO factory/data reset!!)
- connect to your phone with "adb shell" (while in recovery!!)
- in the shell execute following commands:
(do this only if you have active app2sd and your app/app-private/dalvik-cache directories are on the SD card EXT partition)
mount /data
rm -rf /data/app
rm - rf /data/app-private
rm - rf /data/dalvik-cache
mkdir /data/app
mkdir /data/app-private
mkdir /data/dalvik-cache
chmod 777 /data/app*
chmod 777 /data/dalvik-cache
chown 1000:1000 /data/app*
chown 1000:1000 /data/dalvik-cache
- now we will copy back the apps on the nand
mount /sd-ext
cp /sd-ext/app/* /data/app
cp /sd-ext/app-private/* /data/app-private
(no need to copy the dalvik-cache content!)
umount /sd-ext
- so now we have all our app on the nand again. next step is to put the 04data2extV4 script into the right place
umount /data
mount /system
exit
(now leaving shell to copy file)
in the command prompt execute:
adb push pathtothescript/04data2extV4 /system/etc/init.d/
adb shell
(back to shell again)
chmod 755 /system/etc/init.d/04data2extV4
chown 0:2000 /system/etc/init.d/04data2extV4
rm /system/etc/init.d/05mountsd
(remove script which mounts SD card differently)
rm /system/etc/init.d/10apps2sd
(remove script which makes app2sd)
umount /system
exit
(now leaving shell)
now we have to repartition your SD card to 2 ext and 1 fat partition. For this use the utility1 I uploaded which is modified version of the original utility in /sbin.
in command prompt:
adb push pathtoyourscript/utility1 /sbin/
adb shell
(back to shell)
chmod +x /sbin/utility1
(now execute the utility)
/sbin/utility1
choose 0 from menu: Partition SD 2 EXT 1 FAT (can set sizes)
enter the first EXT partition size (min. 3-400, I use 1300 - values are in MB)
enter second EXT partition size (min. 2-300, I use 300 - values are in MB)
the rest will be configured for the FAT32 file system
On the first EXT partition you will store the whole /data
On the second EXT partition you will store the /data/data in a loop file
Now we are hopefully ready to reboot, the rest will be managed by the 04data2extV4 script, wich creates the ext2 file systems, converts them to ext4, stets file system option such as journaling where needed, etc and copies everything from nand to ext4.
I hope this works for all. I reached Quadrant score for total system score about 455, without data2ext I had about 400. But I have a Class4 SD card. On class6 I'm sure it will be even faster.
Again, it is risky, so take care.
The boot process can be long, because at the first time it will copy the whole /data directory to the first EXT partition, the data/data directory on the 2nd partition, and will recreate the whole dalvik-cache (remember we wiped!).
Enjoy
Regards,
byr2
PS: in the RAR file you will find an FSTAB file. I removed the /sd-ext mount option from it. If you have trouble, try to replace the fstab file in /etc to this fstab file or remove the /sd-ext mount option from your fstab

Thank you for this.
Just a few questions. The EXT partitions, are they EXT 2, 3 or 4?
Are you using exactly the same script as ownhere? As in, you use exactly his method, aka the following one:
"I would like to share my Data2EXT script, a collection of data2sd(Thanks to sibere) and the advantages of ext4, you can let the phone run at peak efficiency.
I use EXT4 for whole SD 2nd partition, and create a ext2 loopback device with 1/5 size of 2nd partition.
The files in /data/app,/data/.systemapp it not updated frequently. so store them in EXT4.
In /data/data and /data/app-private, a lot of small files and sqlite database, is updated frequently. so store thenm in EXT2 loopback device.
/data/dalvik-cache is stored in phone memory /mnt/asec/mtddata for distributed IO pressure.
/data/misc, /data/system, /data/property is stored in phone memory /mnt/asec/mtddata for system settings restore(or you will lost your timezone/language settings after reboot).
use this script, I can get quandrant score:2000+, IO:4000+, my phone is smooth and quickly."
Are you running this stable? Reboots? All apps show after reboots?

Hi!
The two EXT partition is EXT4 partition. The script I provided is based on ownhere's script, wich is modified only in a few places.
For example on hero I do not have a /system/xbin/mkfs.ext4 like he had, but only a mkfs.ext2, so I have to create the EXT4 partition on a different way. Also the loopback device is set up in a different way than in the original script.
But what you quoted remains so, which means I did not modified any location of storage files, nor did modified filesystem parameters. First EXT4 ha no journaling ,second EXT4 has journaling, ext4 loop device has no journaling.
You can diff the two scripts to see the difference, but an example here:
Original part:
/system/xbin/mkfs.ext4 -b 4096 -m 0 -F -L userdata /dev/block/mmcblk0p2
/system/bin/tune2fs -o journal_data_writeback /dev/block/mmcblk0p2
/system/bin/e2fsck /dev/block/mmcblk0p2
/system/bin/tune2fs -O ^has_journal /dev/block/mmcblk0p2
Modified part:
/system/xbin/mkfs.ext2 -b 4096 -m 0 -F -L extdata /dev/block/mmcblk0p2
/system/bin/e2fsck -y -v -f /dev/block/mmcblk0p2
/system/bin/tune2fs -O extents,uninit_bg,dir_index /dev/block/mmcblk0p2
/system/bin/e2fsck -fpDC0 /dev/block/mmcblk0p2
/system/bin/tune2fs -o journal_data_writeback /dev/block/mmcblk0p2
/system/bin/e2fsck /dev/block/mmcblk0p2
/system/bin/tune2fs -O ^has_journal /dev/block/mmcblk0p2
As you can see I create an ext2 partition, check it, and with tune2fs I convert it to EXT4, check it again, and set the same parameters ownhere has set.
for loopback device I use:
$BUSYBOX losetup /dev/block/loop1 /mnt/asec/extdata/ext4
$BUSYBOX mount -t ext4 -o loop,sync,commit=3,barrier=1,nouser_xattr,errors=continue,noatime,nodiratime,nosuid,nodev /dev/block/loop1 /mnt/asec/tmpdata
the losetup is needed, because for some reason it won't mount as loop. But this way it is working.
Other parts of the script are untouched.
Regards,
byr2

"Are you running this stable? Reboots? All apps show after reboots?"
Till now everything runs OK, no reboot, all widgets, apps, settings were preserved. After reboot also everything OK. Made some phone calls, no problems so far.
Just with Class4 SD card the speed gain is not as huge as expected. The IO performance is slightly better, but not too much. I've read posts on CDMA Hero forum, where they managed to get quadrant score over 700, but I'm sure not with a Class4 SD card
But the funniest thing in settings is: Available internal phone storage: 1,17Gb

Excuse me for the noob question, but what this process basically does is make the entire phone run off of an ext4 partition on the SD card?

No, not the entry phone, just the /data directory is moved to ext4 partition (wich is accessed frequently by the OS, and where applications and application related data are stored). The kernel, firmware, etc. remains on the NAND.

I wonder what one does when flashing a new ROM =/ You have to go through this procedure every single time? What does AmonRA do when you Wipe Data/Factory Reset?
So essentially the differentiations are due to the fact that we lack a /system/xbin/mkfs.ext4 (since we only have an ext2) and you turn the first EXT4 partition into a non journaling one to fill the lack of that, if i understand correctly.

If you flash a new rom, then all this is gone, since if will overwrite the /system/etc/init.d where our data2ext script is located. So you're right, you have to do this every time you flash a new ROM. A solution would be to pack this script into the ROMs, so it would work out of the box without this hack mentioned in the first post.
The data/factory reset wont work I think, because it deletes from the nand, but since our data/settings are not on the nand but the SD, it will have no effect.
the first ext4 partition is non journaling because ownhere write the script so. You can read in his post why he choosed to do so.
The first EXT pratition where /data is stored is a EXT4 part without journaling
The second EXT partition where the loop file is stored is an EXT4 part with journaling
and the LOOP file which contains an EXT4 partition without journaling
from ownhere's post:
"EDIT:2010/12/12 update V4 script:
/data/:ext4, disable journaling
/data/data:ext4 loopfile on ext4 device, enable journaling on ext4 device, disable journaling on ext4 loopfile, for fast and stable sqlite3 access.
really smooth this time. please trying..."
This is the setup here also.
byr2

I see, the script is essentially the same, with your own critical changes to fit the Hero.
Now what i am trying to understand is what happens when you want to flash a new ROM.
Say you place the 04data2extV4 script in the new ROM's /system/etc/init.d and remove the other two scripts if they exit (05mountsd and 10apps2sd). Does this mean the partitions and this whole process won't have to be repeated?
Also when installing a new rom it is recommended you wipe data and system, if AmonRA doesn't wipe, what do we do?

Well, I've not tested to install a new rom, but let think:
- the partitions are preserved on your SD, so repartitioning is not needed.
- but as you mention all data/settings are currently on your SD, so a wipe does nothing with it, and if you simply put the 04data2extV4 script back to init.d I think it will mess things just up... but if you just delete everything off the two EXT partitions I think it will be fine

That's the only limitation i have so far as to not attempting this.
I wonder if i can mod the AmonRA Recovery to delete the content of those 2 partitions instead =/
I'm going to have a look.

Nice work!
By the way, I was thinking. Why not just make an ext partition, add a line for it in fstab and set the mountpoint for it to be /data. What would happen if I tried this?

Byr2.... just wanted to say this is first thing i read today and without coffee all I heard was
Blah blah blah dangerous blah blah blah faster blah blah risk blah blah even faster
Lmao
Once I get my head working it should prove interesting though
Thanks for this experiment
Sent from my HTC Hero using Tapatalk

ok, sitting down at pc now, drinking espresso...
this is starting to sink in
Instant benefit is huge storage space which, coupled with the method used, ought to have a general performance improvement over previous ext-based apps2sd, right? if so then removing ext-based apps2sd and using this script concept instead seems like a good plan if, for example, every rom builder switched to this concept and removed old apps2sd stuff simultaneously. Does that sound right or wrong?
if you have a fast sd card then the speed benefit should be very good, better than using ext-based "apps2sd" since it's the actual data partition on the ext card.
and I too would like to know the answer to the brief question someone already asked - what about just setting a mount point to data on the sd card and doing it that way in simple linux fashion? I'm not bothering to think this through - coffee's still kicking in - I'll leave the intelligence to you folks who are already awake.
IMHO this idea of yours is quite brilliant as a goal (well ok, it's not your idea, but you did it first on the gsm hero . The thing that slows my phone the most is reads and writes to phone memory and to my stock sd card. I can upgrade the sd card but the phone memory writes are still slow. This would improve the speed of the largest file writes when managing my apps and installing from the market which we all know brings our phones to a standstill briefly during installs and so on.
How's the performance when installing apps? does your phone still crawl for a bit at those crucial moments or is it all somewhat less of a dramatic slowdown now?

Hi All!
Well, If you do not change the ROMs frequently, it should not be a problem that ROM developers do not include this script. However till now only one ROM developers have taken care about this opportunity, I think it was Cronos. They have a modified ROM wich supports this kind of functionality.
The speed gain over app2sd comes from storing the whole data partition on a faster SD card than the internal NAND. As my tests show a class4 SD card is slightly faster than the internal NAND (not much, say about 5-10%). A class6 SD card could be much more faster. All application access the data partition to read and write settings/data. Even the Frimware stores things there, so if you have a fast SD card, overall system performance could be great.
Not to mention about stored data space. With app2sd you only store the downloaded apk-s on the EXT. With data2ext you have extra space for the application data/settings/cache wich size is determined by you and your SD card size ! (See screenshot!)
A simple fstab modification could work, however the script does do a lot of things beside copying everything to EXT. It also sets up some symlinks during every boot, so a part of the script is needed always. Also there is 2 EXT partitions, and the /data/data is stored on a 2nd EXT partition in an EXT4 loop file image So it's not that easy to simply mount it via fstab.
byr2

dkelley said:
Byr2.... just wanted to say this is first thing i read today and without coffee all I heard was
Blah blah blah dangerous blah blah blah faster blah blah risk blah blah even faster
Lmao
Click to expand...
Click to collapse
Hahaha. I've read this several times now and still all I get is what you wrote above.

I got this after all
Code:
$mount
rootfs on / type rootfs (ro)
tmpfs on /dev type tmpfs (rw,mode=755)
devpts on /dev/pts type devpts (rw,mode=600)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
none on /acct type cgroup (rw,cpuacct)
tmpfs on /mnt/asec type tmpfs (rw,mode=755,gid=1000)
none on /dev/cpuctl type cgroup (rw,cpu)
/dev/block/mtdblock3 on /system type yaffs2 (ro)
/dev/block/mtdblock4 on /cache type yaffs2 (rw,nosuid,nodev,noatime)
/dev/block/mtdblock5 on /mnt/asec/mtddata type yaffs (rw,nosuid,nodev,relatime)
/dev/block/mmcblk0p2 on /data type ext4 (rw,nosuid,nodev,noatime,nodiratime,commit=19,barrier=0,nobh,data=writeback)
/dev/block/mmcblk0p3 on /mnt/asec/extdata type ext4 (rw,nosuid,nodev,noatime,nodiratime,barrier=1,data=ordered)
/dev/block/loop1 on /data/data type ext4 (rw,sync,nosuid,nodev,noatime,nodiratime,commit=3,barrier=1,data=writeback)
/dev/block/vold/179:1 on /mnt/sdcard type vfat (rw,dirsync,nosuid,nodev,noexec,uid=1000,gid=1015,fmask=0702,dmask=0702,allow_utime=0020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
/dev/block/vold/179:1 on /mnt/secure/asec type vfat (rw,dirsync,nosuid,nodev,noexec,uid=1000,gid=1015,fmask=0702,dmask=0702,allow_utime=0020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
tmpfs on /mnt/sdcard/.android_secure type tmpfs (ro,size=0k,mode=000)
All correct?

This is mine:
Code:
rootfs on / type rootfs (ro,noatime)
tmpfs on /dev type tmpfs (rw,mode=755)
devpts on /dev/pts type devpts (rw,mode=600)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
none on /acct type cgroup (rw,cpuacct)
tmpfs on /mnt/asec type tmpfs (rw,mode=755,gid=1000)
none on /dev/cpuctl type cgroup (rw,cpu)
/dev/block/mtdblock3 on /system type yaffs2 (ro)
/dev/block/mtdblock4 on /cache type yaffs2 (rw,nosuid,nodev,noatime)
/dev/block/mtdblock5 on /mnt/asec/mtddata type yaffs (rw,nosuid,nodev,relatime)
/dev/block/mmcblk0p2 on /data type ext4 (rw,nosuid,nodev,noatime,nodiratime,commit=19,barrier=0,nobh,data=writeback)
/dev/block/mmcblk0p3 on /mnt/asec/extdata type ext4 (rw,nosuid,nodev,noatime,nodiratime,barrier=1,data=ordered)
/dev/block/loop1 on /data/data type ext4 (rw,sync,nosuid,nodev,noatime,nodiratime,commit=3,barrier=1,data=writeback)
/dev/block/vold/179:1 on /mnt/sdcard type vfat (rw,dirsync,nosuid,nodev,noexec,uid=1000,gid=1015,fmask=0702,dmask=0702,allow_utime=0
020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
/dev/block/vold/179:1 on /mnt/secure/asec type vfat (rw,dirsync,nosuid,nodev,noexec,uid=1000,gid=1015,fmask=0702,dmask=0702,allow_ut
ime=0020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
tmpfs on /mnt/sdcard/.android_secure type tmpfs (ro,size=0k,mode=000)
/dev/block/dm-0 on /mnt/asec/com.qualcomm.qx.neocore-1 type vfat (ro,dirsync,nosuid,nodev,noexec,uid=1000,fmask=0222,dmask=0222,code
page=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
Seems your's almost the same...
The key is this:
Code:
/dev/block/mmcblk0p2 on /data type ext4 (rw,nosuid,nodev,noatime,nodiratime,commit=19,barrier=0,nobh,data=writeback)
/dev/block/mmcblk0p3 on /mnt/asec/extdata type ext4 (rw,nosuid,nodev,noatime,nodiratime,barrier=1,data=ordered)
/dev/block/loop1 on /data/data type ext4 (rw,sync,nosuid,nodev,noatime,nodiratime,commit=3,barrier=1,data=writeback)
How is your phone performing? Was everything OK?
byr2

Everything seems to be ok )
There is no more sticking when new apk installed. Quadrant score around 500.
I have Class 10 sd card (prooflink).
But I have had a little problem - not enough nand space to hold all of my apks, so I've decided to push most of them via adb from my laptop after Hero's reboot.
Quadrant score not so high as I expected, but anyway I'm happy =)

if you push it via ADB after reboot, than it should get to the right place
The quadrant score is affected by 3D and CPU also, but can you feel the "speed" while testing the IO? Does it run fast? How long is the DB read/write, FS read/write?
I'm happy someone else could get this thing working
byr2

Related

Applications to your SD card ?

Hi guys,
sorry because this has probably been answered somewhere but I couldn't find it ....
So here it goes: I just installed Modaco's ROM and it works perfectly. Now I would like my apps to install directly on the SD card.
What I understand is that I need to create a EXT3 partition but I don't know what the hell that means, nor how to do it.
I just put my SD card in the hero, didn't format anything ....
Dould you give a step by step guide ? On how to create that EXT3 partition, how to use A2SD afterwards ? Also, what to do with the currently installed applications ?
Thanks
Geeum
Dould you give a step by step guide ?
Click to expand...
Click to collapse
http://androidandme.com/2009/08/news/how-to-manually-partition-your-sd-card-for-android-apps2sd/
how to use A2SD afterwards?
Click to expand...
Click to collapse
Just use MoDaCo's custom ROMs. (It will do it automatically)
can app2sd work with official ROMs not rooted?
Hi guys I finally partitioned my sdcard to have an ext3 partition.
I thought A2SD was automatic but apparently applications don't go to the sdcard: my phone memory has changed after I installed an app ...
Here is what I get if I type the 'print' command on parted
Code:
(parted) print
print
print
Model: SD 00000 (sd/mmc)
Disk /dev/block/mmcblk0: 8166MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Number Start End Size Type File system Flags
1 512B 7110MB 7110MB primary fat32 lba
2 7110MB 8134MB 1024MB primary ext3
3 8134MB 8166MB 32.3MB primary linux-swap(v1)
So I DO have ext3. Then I found this post that explains how to check if A2SD is working:
Thanks! Here is my way to check if a2sd is working:
Check if the ext3 partition exists (please see androidandme.com for details):
Code:
# mount | grep mmcblk0p2
mount | grep mmcblk0p2
/dev/block/mmcblk0p2 on /system/sd type ext3 (rw,noatime,nodiratime,errors=continue,data=ordered)
Compare available/used space on mmcblk0p2:
Code:
adb shell
# df
df
Filesystem 1K-blocks Used Available Use% Mounted on
tmpfs 98520 0 98520 0% /dev
tmpfs 4096 0 4096 0% /sqlite_stmt_journals
/dev/block/mtdblock3 174080 153460 20620 88% /system
/dev/block/mtdblock5 169728 27240 142488 16% /data
/dev/block/mtdblock4 133120 1160 131960 1% /cache
/dev/block/mmcblk0p2 468331 11767 431579 3% /system/sd
Now install an app from the market (as suggested by fleming222) and again run "df". Compare the available (or used) space on /dev/block/mmcblk0p2 with the first "df".
Click to expand...
Click to collapse
This is what I get (I can see on the first line that my ext3 is not listed ... weird)
Code:
/ # mount | grep mmcblk0p2
mount | grep mmcblk0p2
/ # df
df
Filesystem 1K-blocks Used Available Use% Mounted on
tmpfs 98508 0 98508 0% /dev
/dev/block/mtdblock4 133120 1496 131624 1% /cache
HELP !
By the way I'm running MODACO's 2.6 custom ROM ...
Geeum said:
By the way I'm running MODACO's 2.6 custom ROM ...
Click to expand...
Click to collapse
The best way to do it if your not too good with the adb commands is to install the AMON recovery image for Hero.
It has all the options to create the correct sd card partitions from the recovery mode, then just install your custom rom and everything should work.
You will need to format your sd card so you may want to create a backup of it. And I would suggest that you wipe the phone clean as well. Once you install the modaco rom it automatically moves all of your apps to the sd card.
The link for the recovery image is here:
http://forum.xda-developers.com/showthread.php?t=561124
You will also need Instant Root (http://neilandtheresa.co.uk/Android/), which you should install first, it is an .apk file so you will need a file manager for your phone. It should pop up with a notification "instant root was successfully" or along those lines. (If you have a problem installing Instant Root make sure Settings>Applications>Unknown Sources is checked)
You should then place the AMON recovery image onto the sdcard making sure it is not put into any folders (root).
Instant root then allows for root access with adb shell using the following 2 commands:
1) adb shell
then
2) su
the icon should change from a $ to #.
Now flash the recovery image with the following command:
flash_image recovery /sdcard/recovery-RA-HERO-v1.2.3.img
That should do it, remove the usb and battery and once you have replaced the battery boot into recovery mode.
I ran the following commands in this order from recvoery mode:
- Wipe data + cache (factory reset)
- Wipe ext partition on SD-card
- Wipe Dalvik-cache on SD-card (NOT on /data !) [email protected] ALWAYS WORK
- Partition SD-card : 500MB Ext2, 32MB Swap, remaining Fat32
- Convert ext2 to ext3
- Installed ROM
Hope this helps
Hi and thanks for your reply. I finally sorted my problem, I was typing the commands from the recovery console, and now my ext3 shows:
Code:
# mount | grep mmcblk0p2
mount | grep mmcblk0p2
/dev/block/mmcblk0p2 on /system/sd type ext3 (rw,noatime,nodiratime,errors=conti
nue,data=ordered)
I checked the space with df, and I can see it changing when installing apps.
However, so does the 'internal memory of the phone' in the settings panel; it keeps getting smaller as I install apps
Why is that ?
really want to do this but i find it all to hard and confusing
Geeum said:
I checked the space with df, and I can see it changing when installing apps.
However, so does the 'internal memory of the phone' in the settings panel; it keeps getting smaller as I install apps
Why is that ?
Click to expand...
Click to collapse
I think this is inevitable as the app uses some part of the phone memory for cache files etc (I am not 100% certain on this, just saw it written in an older post on the same topic).
I do not know if this is a coincidence but for some reason I have tried partitioning manually (the way your doing it) in the past and have recently used the AMON recovery image for partitioning instead. I notice that the recovery image tends to use less of the internal storage for each installed app, I don't know why but you could try it out yourself.
guys, i am using modaco's 2.6 rom. so if i just create partitions this custom rom will automatically install newly installed apps to the sd card. right?
but what if i already got many apps installed? will those be deleted while creating partitions?
is there any way so that i can backup the whole application folder and restore after creating partitions?
cheers
apparently if you do ext2 + swap + fat32, you can't do a Nandroid backup (it keeps giving errors). anyone got similar issues?
kazuni said:
apparently if you do ext2 + swap + fat32, you can't do a Nandroid backup (it keeps giving errors). anyone got similar issues?
Click to expand...
Click to collapse
is it so? if so then that wud be a disaster.

Symlinks on 101 G9

i tried to make a symlink on my archos but when i tried it gave Function not implemented.
is there a way to implement this function?
Gr.Roymar
roymar said:
i tried to make a symlink on my archos but when i tried it gave Function not implemented.
is there a way to implement this function?
Gr.Roymar
Click to expand...
Click to collapse
Hi
Where are you trying to create these symlinks?
On the A101S you cannot create symlinks in /mnt/sdcard because this is a vfat partition which does not a support symlinking, you also won't be able to make link in /mnt/storage ; /mnt/data ; /storage ; /sdcard ; This is because these are mounted by the sdcard service which is designed to provide a windows friendly fat view onto the file system.
Creating links anywhere else shouldn't be a problem ( depending on the filesystems you are using )
Hope tjhat helps.
trevd said:
Hi
Where are you trying to create these symlinks?
On the A101S you cannot create symlinks in /mnt/sdcard because this is a vfat partition which does not a support symlinking, you also won't be able to make link in /mnt/storage ; /mnt/data ; /storage ; /sdcard ; This is because these are mounted by the sdcard service which is designed to provide a windows friendly fat view onto the file system.
Creating links anywhere else shouldn't be a problem ( depending on the filesystems you are using )
Hope tjhat helps.
Click to expand...
Click to collapse
Thanx for you're answer.
i was indeed trying to make a symlink in /storage because of de maps of the sygic navigator take a lot of room
Greetz Roymar
roymar said:
Thanx for you're answer.
i was indeed trying to make a symlink in /storage because of de maps of the sygic navigator take a lot of room
Greetz Roymar
Click to expand...
Click to collapse
If you after freeing up space, it is still possible with a bit of hacking voodoo . There are 2 ways I know of in fact.
METHOD 1 - SDCARD SERVICE
you can point the /storage ( /mnt/storage in reality ) to any directory you like. I've currently got mine pointing to an multi partitioned external 32gb sdcard. :0 all you need to do is change the sdcard service in the /etc/init/init.A101S.rc,
Currently it mounts /data/media to /mnt/storage, if you have an external sdcard card handy, try this as a quick and dirty test
From a terminal emulator or adb shell run the following commands
Code:
stop sdcard ;
sdcard /mnt/sdcard 1000 1015 &
ls /storage
You will now see that the external sdcard can be access through /storage. to kill the test run
Code:
pkill -9 sdcard
start sdcard
Everything should be back to normal. As I mentioned to make a more permanent change edit the following line in the /etc/init/init.A101S.rc
Code:
service sdcard /system/bin/sdcard /data/media 1000 1015
METHOD 2 - MOUNT BIND
The other, perhaps more straight forward method uses the bind argument on the mount to command to "mount an already visible filesystem elsewhere" .
In the example I'll bind the external sdcard to a directory in the internal storage
use the terminal emulator or adb once again to execute the following commands
Code:
mkdir /storage/sdcard
mount --bind /mnt/sdcard /storage/sdcard
To make this more parmanent, I would probably create a wrapper shell script for the sdcard service to make sure it all gets started together, that's just my preference, As we say in trade ( or maybe it's a saying i've just made up :silly: ) ; "there's more than one way to mount a file system!"
In Summary: I would go with method 2 as this effectively achieves the desired effect off "symlinking" an external directory in the /storage
Cheers
trevd said:
If you after freeing up space, it is still possible with a bit of hacking voodoo . There are 2 ways I know of in fact.
METHOD 1 - SDCARD SERVICE
you can point the /storage ( /mnt/storage in reality ) to any directory you like. I've currently got mine pointing to an multi partitioned external 32gb sdcard. :0 all you need to do is change the sdcard service in the /etc/init/init.A101S.rc,
Currently it mounts /data/media to /mnt/storage, if you have an external sdcard card handy, try this as a quick and dirty test
From a terminal emulator or adb shell run the following commands
Code:
stop sdcard ;
sdcard /mnt/sdcard 1000 1015 &
ls /storage
You will now see that the external sdcard can be access through /storage. to kill the test run
Code:
pkill -9 sdcard
start sdcard
Everything should be back to normal. As I mentioned to make a more permanent change edit the following line in the /etc/init/init.A101S.rc
Code:
service sdcard /system/bin/sdcard /data/media 1000 1015
METHOD 2 - MOUNT BIND
The other, perhaps more straight forward method uses the bind argument on the mount to command to "mount an already visible filesystem elsewhere" .
In the example I'll bind the external sdcard to a directory in the internal storage
use the terminal emulator or adb once again to execute the following commands
Code:
mkdir /storage/sdcard
mount --bind /mnt/sdcard /storage/sdcard
To make this more parmanent, I would probably create a wrapper shell script for the sdcard service to make sure it all gets started together, that's just my preference, As we say in trade ( or maybe it's a saying i've just made up :silly: ) ; "there's more than one way to mount a file system!"
In Summary: I would go with method 2 as this effectively achieves the desired effect on "symlinked" an external directory in the /storage
Cheers
Click to expand...
Click to collapse
That's a great solution, thank you for the effort
a thanx for you
trevd said:
If you after freeing up space, it is still possible with a bit of hacking voodoo . There are 2 ways I know of in fact.
METHOD 1 - SDCARD SERVICE
you can point the /storage ( /mnt/storage in reality ) to any directory you like. I've currently got mine pointing to an multi partitioned external 32gb sdcard. :0 all you need to do is change the sdcard service in the /etc/init/init.A101S.rc,
Currently it mounts /data/media to /mnt/storage, if you have an external sdcard card handy, try this as a quick and dirty test
From a terminal emulator or adb shell run the following commands
Code:
stop sdcard ;
sdcard /mnt/sdcard 1000 1015 &
ls /storage
You will now see that the external sdcard can be access through /storage. to kill the test run
Code:
pkill -9 sdcard
start sdcard
Everything should be back to normal. As I mentioned to make a more permanent change edit the following line in the /etc/init/init.A101S.rc
Code:
service sdcard /system/bin/sdcard /data/media 1000 1015
METHOD 2 - MOUNT BIND
The other, perhaps more straight forward method uses the bind argument on the mount to command to "mount an already visible filesystem elsewhere" .
In the example I'll bind the external sdcard to a directory in the internal storage
use the terminal emulator or adb once again to execute the following commands
Code:
mkdir /storage/sdcard
mount --bind /mnt/sdcard /storage/sdcard
To make this more parmanent, I would probably create a wrapper shell script for the sdcard service to make sure it all gets started together, that's just my preference, As we say in trade ( or maybe it's a saying i've just made up :silly: ) ; "there's more than one way to mount a file system!"
In Summary: I would go with method 2 as this effectively achieves the desired effect off "symlinking" an external directory in the /storage
Cheers
Click to expand...
Click to collapse
I have archos G9 101 Turbo 250GB HDD
i try with this solution but cant true
app install to internal memory
acasilver said:
I have archos G9 101 Turbo 250GB HDD
i try with this solution but cant true
app install to internal memory
Click to expand...
Click to collapse
I would expect that to be case, This apply's to the SD Versions only, It should be fairly trivial however to change the new location to an hard disk partition. Can run the following adb command and post the output here, then i'll see if I can point you in the right direction :good:
Code:
adb shell mount
Thanks
trevd said:
I would expect that to be case, This apply's to the SD Versions only, It should be fairly trivial however to change the new location to an hard disk partition. Can run the following adb command and post the output here, then i'll see if I can point you in the right direction :good:
Code:
adb shell mount
Thanks
Click to expand...
Click to collapse
Code:
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
C:\Enable_SDE>adb shell mount
rootfs / rootfs rw 0 0
sysfs /sys sysfs rw,relatime 0 0
devtmpfs /dev devtmpfs rw,relatime,size=455588k,nr_inodes=113897,mode=755 0 0
/dev/mmcblk0p2 /mnt/system ext4 rw,noexec,noatime,user_xattr,barrier=1,data=orde
red 0 0
/dev/loop0 / squashfs ro,relatime 0 0
tmpfs /dev tmpfs rw,nosuid,relatime,mode=755 0 0
devpts /dev/pts devpts rw,relatime,mode=600 0 0
proc /proc proc rw,relatime 0 0
sysfs /sys sysfs rw,relatime 0 0
usbfs /proc/bus/usb usbfs rw,relatime,devuid=1000,busuid=1000,listuid=1000 0 0
tmpfs /tmp tmpfs rw,relatime,mode=775,gid=1000 0 0
/sys/kernel/debug /sys/kernel/debug debugfs rw,relatime 0 0
none /acct cgroup rw,relatime,cpuacct 0 0
tmpfs /mnt/asec tmpfs rw,relatime,mode=755,gid=1000 0 0
tmpfs /mnt/obb tmpfs rw,relatime,mode=755,gid=1000 0 0
none /dev/cpuctl cgroup rw,relatime,cpu 0 0
/dev/block/mmcblk0p1 /mnt/rawfs rawfs rw,relatime 0 0
/dev/block/mmcblk0p2 /mnt/system ext4 rw,nosuid,noexec,noatime,user_xattr,barrie
r=1,data=ordered 0 0
/dev/block/mmcblk0p4 /data ext4 rw,nosuid,noatime,user_xattr,barrier=1,data=orde
red,noauto_da_alloc 0 0
/dev/fuse /mnt/nand fuse rw,nosuid,nodev,relatime,user_id=1000,group_id=1015,def
ault_permissions,allow_other 0 0
/dev/block/vold/8:1 /mnt/storage ext4 rw,nosuid,nodev,noexec,noatime,nodiratime,
fmode=0664,dmode=0775,uid=1000,gid=1015,user_xattr,barrier=1,data=ordered 0 0
/dev/block/vold/8:1 /mnt/secure/asec ext4 rw,nosuid,nodev,noexec,noatime,nodirat
ime,fmode=0664,dmode=0775,uid=1000,gid=1015,user_xattr,barrier=1,data=ordered 0
0
tmpfs /mnt/storage/.android_secure tmpfs ro,relatime,size=0k,mode=000 0 0
C:\Enable_SDE>
doesn't work on CM10.1
Hello,
Thank you for your methods!
I used your method 2 on Blue ROM (4.0.4) with success but I just put the CyanogenMod 10.1 and it doesn't work anymore, here is the command line I put on my script for CM10.1 :
mount -o bind /storage/sdcard1/Android /sdcard/Android
It only works on ADB command from the PC, if I try on Terminal emulator on the tablet I have no error but no effect.
Do you have a solution please?
pourmadeesse said:
Hello,
Thank you for your methods!
I used your method 2 on Blue ROM (4.0.4) with success but I just put the CyanogenMod 10.1 and it doesn't work anymore, here is the command line I put on my script for CM10.1 :
mount -o bind /storage/sdcard1/Android /sdcard/Android
It only works on ADB command from the PC, if I try on Terminal emulator on the tablet I have no error but no effect.
Do you have a solution please?
Click to expand...
Click to collapse
Yeah that sounds about right, it's because the /storage directory on 4.2.2 ( CM 10.1 ) is just a "virtual directory" the real data is /data/media/0 try changing your bind to there, You can probably just get away with a symlink especially if you're external sdcard is in ext4... I'll have to have a play with that!! :good:
Thank you, it works from Terminal Emulator!
An another linked question :
I restored my Gameloft games (Ice Age Village, MIB,... and Rayman) from Titanium Backup as usually but this time I have problem once I put this symlinks, it tells me I need to download datas I select Yes or ok but I have an error and I can't play, I precise it's only with CM10.1 with symlinks, I did it many time with 4.0.4 with no problem
pourmadeesse said:
Thank you, it works from Terminal Emulator!
An another linked question :
I restored my Gameloft games (Ice Age Village, MIB,... and Rayman) from Titanium Backup as usually but this time I have problem once I put this symlinks, it tells me I need to download datas I select Yes or ok but I have an error and I can't play, I precise it's only with CM10.1 with symlinks, I did it many time with 4.0.4 with no problem
Click to expand...
Click to collapse
Hmm, Maybe the Symlink method is no good for you in this particular instance. Just to be clear are you saying you used "mount --bind" or did you go with a symlink using the "ln -s" command... At a wild guess I'd be inclined to thing it maybe be some anti-piracy measure in Gamesloft Games or it could be a permissions issue.
Just a little tip for you. If you're going to ask questions you're going have to bring folks more than "I've got an error" That's doesn't tell anyone anything really.... What does the error say? for starters. The more info you have the better. Then the people who want to help can help and you get to play you're games a whole lot sooner :good: Winners All round :silly:
trevd said:
Hmm, Maybe the Symlink method is no good for you in this particular instance. Just to be clear are you saying you used "mount --bind" or did you go with a symlink using the "ln -s" command... At a wild guess I'd be inclined to thing it maybe be some anti-piracy measure in Gamesloft Games or it could be a permissions issue.
Click to expand...
Click to collapse
I did this command :
Code:
mount -o bind /storage/sdcard1 /data/media/0
My goal is to get "data" and "obb" at least on the sdcard instead of internal memory, am I doing the right thing?
Just a little tip for you. If you're going to ask questions you're going have to bring folks more than "I've got an error" That's doesn't tell anyone anything really.... What does the error say? for starters. The more info you have the better. Then the people who want to help can help and you get to play you're games a whole lot sooner :good: Winners All round :silly:
Click to expand...
Click to collapse
You're allright, here is some two screenshot of MIB without symlinks :
If first ask for additionnal files, I select Yes and it starts downloading.
But, if I run the mount command, after the first screenshot asking additionnal datas, I get this message (and screenshot doesn't work, "Couldn't capture screenshot. Couldn'tsave screenshot. Storage may be in use.") : "A problem occured and the download could not be completed successfully. Do you want to restart the download? An unstable network may cause problems when downloading." I select Yes but additionnal datas message and the same message bootloop. It seems it's a storage problem...
For Rayman, it starts without problem without symlink and I get a "Download error, please make sure you are connected to Internet and try again later" with mount command. Of course I'm connected...
Hope you can help me
Nobody knows what is the problem with symlinks and CM10.1 ?

[DEV][19NOV] Native Mount DataOnEXT with DalvikOnNAND (Test #2)

Index
Post 1 - Android Boot Process, Space Restrictions in /data, Usual Workaround scripts that load from init.d
Post 2 - The Permanent Solution for Internal space issues in /data, and how it is executed (Step by Step)
Post 3 - Credits, Benefits, Note on Cache, Note to ROM Makers on ClockWorkMod Updater Script, FAQ
Post 568 - Test ROM for Native Mount DataOnEXT with DalvikOnNAND
Post 622 - Test #2 ROM for Native Mount DataOnEXT with DalvikOnNAND
Thanks to quite a lot of inputs, ideas, testing and feedback, I have been able to give a working solution to reliably mount /data to the EXT partition in our HD2’s SD Card before Android initializes. I am redoing the first three posts of this thread to give a complete info that answers the what, why, how questions.
Before I even start, let me make my objectives clear:
Have a system partition that is as big as I want in NAND, so that I can install any bloated ROM I want without chopping down anything, and yet have a /data partition for all my apps and their bloated data
Avoid using internal (NAND) userdata partition completely, so that if the day comes where my /system is full of bad blocks, I will still be able to create a system partition in the ~900+ MB internal NAND my TMoUS HD2 has
With this method, this is how I am using my phone:
Device - TMoUS HD2
Boot Loader - cLK 1.5.0.9
Partitions (in order) and Sizes - Recovery - 10 MB, Boot - 10 MB, System - 500 MB, Userdata - <I didn't bother >, Misc - 1 MB, Cache - 2 MB.
Recovery - CWM Based Touch Recovery B8
SD Card - 16GB Transcend Class 6 with 4 GB EXT4 Partition (mounted as /data, of course!)
My idea for this setup is that a user should not be having to get into the boot loader or changing partition sizes for every ROM. With this setup, I am having a close-to-native-Android device setup and can flash any ROM I want, without worrying if my partitions will accommodate it.
So what exactly does this method do? Before I answer that, a brief on the Android booting process – Note that this is simplified to meet this thread’s requirement, and not necessarily completely accurate. When you power on the phone the following actions happen…
The Android Boot Process
Bootloader – In HD2’s case, Magldr or cLK – loads the kernel based on how you have configured the phone.
Kernel – The kernel (zImage) is loaded into RAM along with an initial ramdisk (initrd.gz), which initializes various devices (IO, memory, GPU, etc.), interrupts, and mounts the root file system (/). After this, the first user-space process called init is started.
Init – this is a binary file that is contained within the initrd.gz. The init binary processes init.rc and init..rc , along with other .rc files that are called by these two .rc files. Some of the key functions (from this thread’s perspective) in the order of their initialization/ execution are:
The init process follows the instructions in the init.rc and init.xyz.rc files and creates empty directories including /data. It then mounts the storage devices (partitions in the internal NAND (MTD)) to these empty directories. The NAND partition for system is mounted to /system, followed by the partitions specified for data, cache, etc. The directories for dalvik-cache (/data/dalvik-cache) are also created by the init process after mounting the specified device to /data.
The init process then starts various services including adb, service manager, Volume Daemon (vold) for media like SD Card (FAT partition). Most importantly, the zygote service which initiates the Dalivk-Cache is loaded in this sequence.
As we all know, Android is based on Linux. The boot sequence described above is common for all Linux machines – until the zygote stage. Core Android file like core, framework, services, IME, policy, etc. are executed from the Dalvik-Cache and hence Initialization of the Dalvik Cache is pretty much where Android comes into the picture
The sysinit/ run-parts part, which runs scripts from the /system/etc/init.d later the Zygote stage. No matter how this is done, Android has already started loading by the time the boot process comes to executing scripts in /system/etc/init.d
Now that you know what happens when Android boots, you may also have realized that mounting a device to /data happens early in the boot process.
Space Restriction in HD2, and the twist with bad blocks
The 'normal' way to store all OS data (dalvik-cache, Configurations & Settings, Accounts, etc.) and User data (Apps, app data, mails and messages, call logs, etc.) is to have a partition named userdata in the HD2’s NAND, and have it mounted to /data during the init process. The drawback in using the userdata partition is that the space you can have for userdata is inversely affected by the size of your system partition, and whether your phone is an EU/ International HD2 or a TMoUS HD2.
Best case is, if you have a ~100 MB system partition for a really scaled-down, space-optimized OS, you will get about 800+ MB for userdata partition on TMoUS and ~400 MB on an International/ EU HD2. This space might be good enough for a lot of us who only have light-to-medium use of our phones. But for quite a lot of us, this is a bottleneck. As if that is not enough, we have these villainous bad-blocks that creep up slowly, determined to swallow up the internal NAND completely, though it would be several years down the line before that happens
The Workaround
The workaround for this method is to have a script in the /system/etc/init.d that mounts the EXT partition in the SD Card to sd-ext, moves a part of /data from the userdata (NAND) to the EXT partition (sd-ext) and creates symlinks (symbolic link) in /data (userdata) to point to the new location in the EXT partition. In this method you will not see an increase in what the phone shows for internal space. The apps you install and then some more will be installed in the EXT partition, which will be symlinked as /data/app and so on.
The other way is to move the mount point (and the contents) for the /data directory from userdata (NAND) to the EXT partition in the SD Card, mount the userdata (NAND) as sd-ext or some other directory, create dalvik-cache (and maybe /data/data - where all user apps' data are stored) in the userdata and symlink that location back to the EXT partition (which is now /data). In this method, you will get the size of your EXT partition as the internal space. Moving parts of /data to NAND will in theory give you better performance, as the most read/ written part of the phone resides in NAND.
The benefit of these methods is that it is very simple and easy to install on any rooted ROM through the recovery. This method balances performance by moving the static part of /data (like the .apk you install) to the EXT partition, keeping the dynamic part (/data/data – where all app data are stored, or /data/dalvik-cache – from where all apks are executed) in NAND.
The drawbacks with this workaround:
This method is initiated AFTER Android boots. No matter how it is executed or what it does, anything that loads from /system/etc/init.d is executed only after at least some parts of Android loading process is initialized. This leads to weird outcomes – low sound issues, unpredictable behavior of the OS or some apps (Notification Widget toggles not working, etc.), and then some more. There is no reasonable way to predict what issues you may get. In fact you cannot be sure if the issue is with the ROM or if the issue is occurring because of the move2sd script you are using. If you are facing an issue that none others are facing and if you have a move2sd script, then you MUST undo what the script does before you can start troubleshooting.
In some cases, an in place upgrade (upgrading from one version of the same ROM to a higher/ newer version, or upgrading from one ROM to another ROM of the same Android version/ build) will corrupt the apps' data as well as configuration, etc.
Moreover, one move2sd script that works fine with one ROM will not work exactly the same way with another ROM or even an updated version of the same ROM.
There is always some amount hit or miss involved in this workaround!!!
The Solution
…is what this thread is all about. Like we saw, /data directory is mounted to a storage device pretty early in the boot process – so early that it is only Linux at the time /data gets mounted and Android is still yet to be loaded. The storage device that is traditionally mounted to /data directory is the internal NAND’s userdata partition.
The simple solution I have for unrestricted and reliable internal space is to directly mount the EXT partition of the SD Card to /data during the Linux init process (instead of first mounting the NAND’s userdata, loading Android, and moving the /data mount point or some contents within /data to the EXT partition while Android is loading).
How is this done?
The Easy way - Thanks to @Kokotas (Note that I have not updated Kokotas' .zip file to include the recent changes (20th October 2012) (See The Long Way below)
Visit the post (quoted below), thank @Kokotas, download the .zip from the post, flash it in recovery and lets us know how it goes. Did I tell you to make sure you have a NANDROID backup?
This .zip can be used to install DataOnEXT mod on the fly - your ROM will be the same when you reboot after flashing this .zip, except that /data will be mounted from the EXT partition with all your apps, settings, etc. intact.
kokotas said:
If you have a normal NAND installation(and an ext2/3/4 partition on your card) and want to give ph03n!x's DataOnEXT method a try,
BUT you think it is difficult THEN you can try the attached recovery package.
It follows all the steps of the described process AND at the end it copies all of your /data to the ext2/3/4 partition. <= This means that after rebooting, you'll have your system set up as before but with DataOnEXT!
And if you want to go back to your previous setup (that is a normal NAND installation without DataOnEXT), just flash the original kernel to your device.
Have in mind that I have only tested it on my phone. If you do try it, please leave a comment. Especially MAGLDR users! Cause it is not tested with MAGLDR.
Thank you ph03n!x for a great concept!
Click to expand...
Click to collapse
The Long Way (or this is how I started off doing this mod )
The following steps show how to modify a ROM’s boot process to mount the EXT partition as /data.
IMPORTANT: THIS PROCESS INVOLVES USING DISXDA’s KITCHEN in CYGWIN. I EXPECT YOU TO BE ABLE TO SETUP CYGWIN AND USE DSIXDA KITCHEN. EVEN WHILE I AM TRYING TO GIVE A STEP-BY-STEP HERE, I WILL NOT ANSWER QUESTIONS ON HOW TO SETUP AND TROUBLESHOOT DSIXDA’s KITCHEN OR CYGWIN.
Assumption:
For the sake of simplicity, I am going to assume that you have a ROM that is installed in your phone, and you want to modify the same ROM for natively mounting EXT to /data.
I am also going to assume that your HD2 has a SD Card in it, and the SD Card has been partitioned in such a way that the first partition is the FAT partition that Windows sees, and the second partition is a EXT2/3/4 partition. You may or may not have a Swap partition, but I strongly recommend NOT enabling Swap.
Prerequisites:
Take a NANDROID backup using ClockWorkMod
Back up all user apps and system app data using Titanium Backup
Install Cygwin
Install @dsixda’s Kitchen
Install Notepad++ (or any other UNIX compatible text editor)
@dsixda’s Kitchen – Setting up a working folder
While these steps will involve the entire ROM, note that you will only using the boot directory or boot.img for this mod.
You need a rom.zip file that has at least these three directories when you open the .zip – META-INF, system, boot (or boot.img)
If your rom.zip file has a different structure, just extract the three directories alone from it and create a new zip. Essentially, when open the rom.zip you should see only these three directories
An Example:
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Place this zip file in the original_update directory under Kitchen. A sample directory structure of the Kitchen is as follows:
From Command Prompt, run cygwin.bat, browse to the directory where you have installed the kitchen and run it using ./menu
Select option 1 for setting up a Working Folder, follow instructions. Congratulation! You should have a working folder now. After the Working Folder is setup, navigate to the WORKING_091712_123456 directory. Make sure you have a directory named boot, or a boot.img here. To be double sure, you can replace the boot directory or boot.img with the one from the ROM you want to modify and flash.
Steps for mounting EXT to /data
Back in the Kitchen Menu, select 0 for Advanced Options
Select 20 for Tools for boot image
Hit z for Extract kernel+ramdisk from NAND boot folder (or w for Extract kernel+ramdisk from boot.img)
Now navigate to the BOOT-EXTRACTED directory that got created under the directory where you installed the kitchen. You will see something similar to this:
Edit init.rc and init.htcleo.rc to find all mentions of sd-ext and comment them out with # in the begining. This step is to ensure there is no directory called sd-ext when your phone boots, avoiding any confusions
In init.rc, look for the on fs part. Under this part there will be a line that reads mount yaffs2 [email protected] /system noatime nodiratime. Below this will be another line that reads mount yaffs2 [email protected] /data nosuid nodev noatime nodiratime. Comment this line out with a # (#mount yaffs2 [email protected] /data nosuid nodev noatime nodiratime). Also, if you have mount yaffs2 [email protected] /system ro remount, comment that out too (make it #mount yaffs2 [email protected] /system ro remount)
Do all the following changes AFTER mount yaffs2 [email protected] /system noatime nodiratime, else the process will fail.
Code:
# Wait to ensure SD Card is available, and scanning file system of EXT
wait /dev/block/mmcblk0p2
wait /dev/block/mmcblk0p2
wait /dev/block/mmcblk0p2
wait /dev/block/mmcblk0p2
exec /system/xbin/busybox sh -c "/system/bin/e2fsck -f -p -t -t -v /dev/block/mmcblk0p2 > /system/fsck.log"
# Native Mount DataOnEXT
mount ext4 /dev/block/mmcblk0p2 /data wait nosuid nodev noatime nodiratime barrier=0 data=writeback
Leave everything else in this section as is - you will have another line to mount cache, do not remove or modify it.
Next, scroll down to the section that reads on boot. Add the following like under it:
Code:
# SD Card Cache
write /sys/devices/virtual/bdi/179:0/read_ahead_kb 2048
Build the NAND boot folder (for Magldr yboot) or boot.img (for cLK / Magldr rboot) as per your requirements
Head back to the Working folder, copy the boot directory or the boot.img and include it in your original ROM’s zip file (not the one you created for the kitchen, in case you had to). Check the init.d directory of the ROM’s zip to ensure there is no script that mounts the EXT partition to sd-ext directory, or has a move2sd/ app2sd/ int2sd script – delete such scripts!. Also make sure that your ROM's inatallation does not copy anything to /data during Installation. If it does, move those parts to /system. Or use the instructions I had given under Note to ROM Makers on ClockWorkMod Updater Script to mount the EXT partition as /data in Recovery.
Format your phone’s /system, /data, / sd-ext and /boot from clockworkmod and flash this modified rom.zip
Let your phone boot into Android and settle down.
Configure your Google Account
Download and install Titanium Backup
In Titanium Backup, restore all apps and their data from the backup you had taken earlier
You phone must now have an internal storage that is roughly equal to the size of your SD Card
Note:
I have tested this method with CM7, CM9, CM10 (ParanoidAndroid JB) and Sense ROMs. There has also been feedback that this is working on other AOSP based ROMs like MIUI.
The mount wait command is available in the init binary from CM7 onwards (not sure if it is there in CM6). If you are not sure (or cannot check the source code for init.c)
Misc & Changelog
Credits
All the Devs who are contributing their valuable time to get Android working on our phones
Proz0r for implementing something very similar to what I had wanted. He has essentially implemented the same thing, I am implementing this in a slightly different way, that's all. (To be clear, I did not borrow the idea even though a few others have had the same brainwave that I thought I got first - I only borrowed the part where we have the phone wait until the sd-ext partition is ready for being mounted)
Rick_1995 for the first hint to get this modification to work
Securecrt for showing that this modification is indeed possible
Xylograph for pointing me to another thread where this modification is already done
marco.palumbi - for showing me that I need not have an elaborate script to ensure /dev/block/mmcblk0p2 is available, and a mount... wait option is available in the CM source for init. Also for doing away with the need for extprep script and including the e2fsck routine complete with a fsck.log in init.rc itself.
kokotas - For the recovery-flashable version of this mod that simplifies migrating to DataOnEXT
Some of the benefits of this mod are:
Space - Obvisiouly. The 1GB/ 2GB/ 4GB sd-ext partition will be mounted as internal memory, instead of the usual /data partition from NAND
Reliability - EXT4 partition is arguably more reliable than YAFFS2
Working around Bad Blocks - Most of our phones would be having a bunch of bad blocks by now (lucky you, Hynix NAND folks!). With this method, the /data partition is mounted directly to the EXT partition in your SD Card, and the internal userdata NAND partition is untouched. This means you can create bigger system partitions to work your way around bad blocks. Incidently all the bad blocks my phone has is in the userdata partition - this might be so because that is the area that is written to the most.
Performance - A fast SD Card gives better transfer speeds than NAND (My ParanoidAndroid build is noticeably faster after this modification, with my Class 6 Transcend SD Card with 4GB sd-ext partition)
Simplicity - Don't have to worry about hoards of the a2sd scripts and their variants, and carry-out trial and error as to which ones work reliably with your ROM of choice. /data is transparently and natively mounted to sd-ext, so no more low-sound or other odd issues
/data/data can be in the sd-ext partition, and ROM updates will not corrupt it like in some a2sd scripts
Hassle-free - Whether an EU HD2 or TMoUS HD2, you set a large size (445 MB?!!) for /system and forget about partitioning, as the internal userdata partition is not used at all and can be of whatever size. I am using 300 MB system in both my phones. This is close to native Android phones, where we do not keep changing the system partition size for every ROM that is installed.
A proper multi-user setup - if you had to share the phone with someone else, all you have to do is swap out the SD Card you usually have in the phone to a fresh one that has the required FAT and EXT partitions. The other user will see a fresh Android install and configure whatever they want. When you get the phone back, all you have to do is put your usual SD Card back in, and voila! you have all your stuff just the way it was (Thanks to @teemoo for the tip!)
Trying a new ROM? Make sure you do not botch up your data! Just remove the SD Card you usually use and place one for testing. You can always mod the initrd.gz of your new ROM, flash the boot image or place the initrd,gz in /boot and replace your usual SD Card when you want all your apps, data and settings!
A Note on Cache
As many may remember (and some have pointed out), I had the initial trial with 8 MB of cache for SD Card interface. Over a period of time, I tried various sizes from 512 KB to 16 MB and found out that the cache size is dependant on the phone AND the SD Card that is in it. This experiment involves:
Setting a cache size in the extprep script
Booting from the boot.img using fastboot boot command
Waiting for my phone to connect to Google servers
Opening Root Explorer and confirming the cache size
Running SD Tools, testing the SD Card Read/ Write speeds thrice in a row
Changing the cache size in extprep script, recreating a boot,img, and starting from fastboot boot...
And here are my test results:
As you can see, the higher the cache, higher the difference between the Minimum and Maximum values for Reads. While this test does not consider ALL aspects (like random reads), this gives a good idea of what is the cache size that is the best for my phone and my SD Card. Note that I have two HD2 with 2 different brand/ class of SD Cards, and bot gave pretty different results. You may have to find the right cache size for your phone! Once you got the right cache, even a Class 2 card (the stock Sandisk 16 GB Class 2 that comes with the TMoUS HD2) performa pretty well - no deal-breaking lags even in games like Temple Run or Ninjump.
Also, you should be worried more about read speeds - that is what the phone does most of the time. Write speeds comes into picture mostly when you are installing something. All writes that happen after that are mostly incremental, and never something a major as what happens when you install an app.
Note to ROM Makers on ClockWorkMod Updater Script
While one of the benefits of this mod is that you can set a huge /system partition and forget trimming down anything, some ROMs mounts /data during CWM install to copy certain apps or parts of /system to the /data partition. For such a setup to work well with this mod, the /data partition should be mounted to the EXT partition in the SD Card. I am not sure if there is a straightforward mount command that can be used in updater-script to mount an MMC partition. Thanks to Looki75's post, I have made the script simpler...
The code in updater-script...
Code:
package_extract_dir("work", "/tmp");
format("MTD", "system");
format("MTD", "boot");
mount("MTD", "system", "/system");
mount("ext4","/dev/block/mmcblk0p2","/data");
ui_print("Copying files to /system");
package_extract_dir("system", "/system");
...
...
ui_print("Copying files to /data");
package_extract_dir("data", "/data");
FAQ
I will try and compile some sensible Questions from this thread in the FAQ Section. I request all to read the first three posts at least before asking any questions.
Q: I have an EXT2/ EXT3 partition. You are mounting it as EXT4. What gives?
A: Mounting as EXT4 is backwards compatible with EXT2/3
Q: Where are the initrd.gz that you had posted earlier?
A: I have moved them to this post
Q: Will you modify the initrd.gz from the <name of ROM>?
A: I took more than 6 Hours on a Sunday Night to re-do the first three posts, and give a step-by-step guide. Do you still want me to modify the initrd.gz for you?!! I might post the initrd.gz of ROMs I use or try from time to time, but please don't post requests for modding <name of your ROM>'s initrd.gz. What is that saying about giving a man a fish vs. teaching how to fish...??!
Q: Is ClockWorkMod or 4EXT or TWRP compatible with this mod?
A: Before I answer, note that almost all Recovery tools available for HD2 will mount the SD Card's EXT partition to the sd-ext directory. After doing this mod, /data (in Android) = /sd-ext (in Recovery)
Any backup that includes the sd-ext partition will backup your /data.
At least CWM based recovery will delete dalvik-cache from NAND's data and /sd-ext. After this mod, /data/dalvik-cache (Android) = sd-ext/dalvik-cache (Recovery), so yes deleting Dalvik Cache works
CWN Recovery's Fix Permissions runs in both internal data and sd-ext, so yes that works too
Q: Will installing <name of script/ mod for ROM>, which involves flashing something in Recovery, work after this mod?
A: If whatever you are flashing is limited to modifications in /system, no problems - it will work.
If what you are flashing involves changing or adding content to any part of /data then it will probably not. The reason is when you mount data in Recovery, you will by default mount the internal NAND. After this method, the internal NAND is untouched. You must either modify the updater-script (See Note to ROM Makers on ClockWorkMod Updater Script), or use QtADB/ ADB shell to do what the flashing is supposed to do.
Q: I am getting stuck at boot animation/ getting "Encryption Unsuccessful" message when Android boots...
A: Your ROM's init binary may not support the mount... wait option that I am using. There are other variants like wait <device>, and then mount <device>, see if you can find what it is from the source code of your ROM's init binary (init.c). If all else fails, you may have to include a copy of busybox in sbin, point all commands in extprep script to that busybox, and increase the sleep time using /sbin/busybox sleep 25 (that is an example). In any case, you DO NOT HAVE TO WIPE DATA or WIPE ANYTHING AS SUGGESTED BY THE INSTRUCTIONS IN "Encryption Unsuccessful" message
Q: Can I move part of /data from EXT partition to NAND?
A: You don't have to move anything - that is the beauty of this mod. If, for example, you want dalvik-cache to be in NAND, all you have to do is create a directory, mount NAND to that directory, and symlink /data/dalvik-cache to that directory. Example: In init.rc, make sure /data/dalvik-cache is NOT getting created (comment that line out with a #)
Code:
mkdir /dalvik 0771 system system
mount yaffs2 [email protected] /dalvik nodev nosuid noatime nodiratime
symlink /dalvik /data/dalvik-cache
Q: How do I improve performance?
SD Card interface by design cannot handle multiple read/ write activities simultaneously. An example: Put the SD Card in a card reader attached to your computer. Copy a ~500MB file to it, see what write speeds your get (Win 7 shows an approx value). Now, while the first file is still being copied, copy another similar sized file - what happens to the write speed? It crawls, right?
When you have DataOnEXT, the SD Card is continuously being read from and written to because all your apps, their data, system data and configuration, etc. resides in the EXT partition. You SHOULD NOT expect the same speeds like a full-NAND setup. That said, there a few things you can do to improve performance:
Get a performance SD Card like SanDisk Ultra® microSDXC™UHS-I card
ODEX your ROM
See the previous question's answer for moving dalvik cache to NAND
Increase the NAND's read ahead cache from 4KB to 128 ot 256KB (in init.htcleo.rc)
Set your IO Scheduler to Deadline (use No frills CPU Control)
Q: My ROM slows to a crawl after installing this mod when I install or remove an app...
A: That is normal behavior, let it be for a min or two and it will get back to the usual speeds. This is because installing or removing an app involves creating the DEX file in the dalvik cache, extracting the components (lib files, config files, etc.) for the app and placing them in appropriate locations in /data and so on. This will slow down the system responsiveness because all of these are in the EXT partition, and the SD Card interface is a serial interface that will slow down with multiple simultaneous read/ write requests. However, doing what I have said for the previous question will improve performance quite a bit
FAQ is still WIP, I will keep adding to this section as and when...
ph03n!x said:
Devs - Would be great if you can help me with a way to reliably mount the ext partition to /data in init.rc!!!
Click to expand...
Click to collapse
The problem is with the slow sd cards, Try something of this sort.
Code:
mount ....
while [ $? -ne 0]
do
echo "Waiting for slow sd card!"
sleep 5
mount ....
done
You might have to check a2sd script since I'm not sure what is the success test case for mount, most unix programs return zero so i've based it on that assumption.
Rick_1995 said:
The problem is with the slow sd cards, Try something of this sort.
Code:
mount ....
while [ $? -ne 0]
do
echo "Waiting for slow sd card!"
sleep 5
mount ....
done
You might have to check a2sd script since I'm not sure what is the success test case for mount, most unix programs return zero so i've based it on that assumption.
Click to expand...
Click to collapse
I tried a sleep command before the part where I mount the ext partition to /data, but that doesn't seem to work. I am still figuring out the script language used in Android's init.rc, including a if loop or a whole loop.
Looks like I have to create a service that calls an external script- at least that is how far I have come with my searching
Swyped from my HTC HD2 using xda premium
ph03n!x said:
Space - Obvisiouly. The 1GB/ 2GB/ 4GB sd-ext partition will be mounted as external memory
Click to expand...
Click to collapse
Think you mean internal here, I take it internal space is still used as cache (have an unanswered question in one of Xylo's threads as to why sdcard isn't mounted as cache).
Could you post just the init.rc file to see what modifications have been made?
In terms of loop, while trying to get a squashed system up and running (last squashed rom for HD2 was on gingerbread), the squashed files mounted fine (but couldn't get the system to fully boot, probably a libs issue), the mount command used in init.rc were:
Code:
mount squashfs [email protected]/system/fonts.sfs /system/fonts ro
placed after "on fs".
So modifying for this:
Code:
mount ext4 [email protected]/dev/block/mmcblk0p2 /data rw
, failing that you may need to mount it first as say /mnt/ext then loop.
The issue with the ext mount scripts that I've found (at least with link2sd) is that there is a delay in being granted superuser permissions, using init.rc should overcome that issue.
HypoTurtle said:
Think you mean internal here, I take it internal space is still used as cache (have an unanswered question in one of Xylo's threads as to why sdcard isn't mounted as cache).
Could you post just the init.rc file to see what modifications have been made?
In terms of loop, while trying to get a squashed system up and running (last squashed rom for HD2 was on gingerbread), the squashed files mounted fine (but couldn't get the system to fully boot, probably a libs issue), the mount command used in init.rc were:
Code:
mount squashfs [email protected]/system/fonts.sfs /system/fonts ro
placed after "on fs".
So modifying for this:
Code:
mount ext4 [email protected]/dev/block/mmcblk0p2 /data rw
, failing that you may need to mount it first as say /mnt/ext then loop.
The issue with the ext mount scripts that I've found (at least with link2sd) is that there is a delay in being granted superuser permissions, using init.rc should overcome that issue.
Click to expand...
Click to collapse
I meant internal indeed - corrected it
SD Card as Cache - Not sure if I checked back on the question, but I do remember the discussion. In most recent ROMs, /cache is a part of /data - this is to make space for large apps from the market that may otherwise be limited to the size of the cache partition. Instead of /data, we can have any other writable partition - SD Card or even SD-EXT - but symlinking it to /data seems hassle free and does not require anything more. Linking to SD Card may mean that the SD Card has to always be in the phone, and it should be mounted soon enough.
I haven't checked the loop part you have given - let me do some reading about it (or you can try it and let us know!). I will driving for the next several hours, will post the init.rc later today/ tomorrow...
HypoTurtle said:
Think you mean internal here, I take it internal space is still used as cache (have an unanswered question in one of Xylo's threads as to why sdcard isn't mounted as cache).
Could you post just the init.rc file to see what modifications have been made?
In terms of loop, while trying to get a squashed system up and running (last squashed rom for HD2 was on gingerbread), the squashed files mounted fine (but couldn't get the system to fully boot, probably a libs issue), the mount command used in init.rc were:
Code:
mount squashfs [email protected]/system/fonts.sfs /system/fonts ro
placed after "on fs".
So modifying for this:
Code:
mount ext4 [email protected]/dev/block/mmcblk0p2 /data rw
, failing that you may need to mount it first as say /mnt/ext then loop.
The issue with the ext mount scripts that I've found (at least with link2sd) is that there is a delay in being granted superuser permissions, using init.rc should overcome that issue.
Click to expand...
Click to collapse
ph03n!x said:
I meant internal indeed - corrected it
SD Card as Cache - Not sure if I checked back on the question, but I do remember the discussion. In most recent ROMs, /cache is a part of /data - this is to make space for large apps from the market that may otherwise be limited to the size of the cache partition. Instead of /data, we can have any other writable partition - SD Card or even SD-EXT - but symlinking it to /data seems hassle free and does not require anything more. Linking to SD Card may mean that the SD Card has to always be in the phone, and it should be mounted soon enough.
I haven't checked the loop part you have given - let me do some reading about it (or you can try it and let us know!). I will driving for the next several hours, will post the init.rc later today/ tomorrow...
Click to expand...
Click to collapse
Alright the mount loop command you mentioned and the looping I was discussing with Rick are different. The mount loop command mounts a file (eg: a .iso image) as a device.
The loop I was looking to create is a logical if... else or a while... do loop that will check if /dev/block/mmcblk0p2 is available, else will halt the boot process until it becomes available. The challenge (for me) its that the scripting language used in init.rc its not like the scripts we create for init.d
I am still doing some reading up...
Swyped from my HTC HD2 using XDA Premium
Will this work??
Starting your own service at boot.
...If you have a reason to add something or just what to try things out however it is relatively straight forward to add a native service to that init sequence. I have created a simple example that writes the time elapsed since it started to the log. It wakes up every three seconds. The code for exampleservice.c looks like:
Code:
#define LOG_TAG "Example Service"
#include < utils/log.h >
#include < unistd.h >
int main(int argc, char **argv)
{
LOGI("Service started");
int elapsed = 0;
while(1)
{
sleep(3);
elapsed += 3;
LOGI("Service elapsed time is %d", elapsed);
}
}
Place this somewhere in the platform build system and build as an executable. The build system will place the output in the /system/bin folder on the device. Now all that is needed is to add the following to the init.rc script:
Code:
service exampleservice /system/bin/exampleservice
user exampleservice
group exampleservice
oneshot
and your service will start on init. It is also possible to write your own file using the Android init language that enables your services and variables. Then just add a line to import the file in init.rc and you are done.
Source
Click to expand...
Click to collapse
Will this method work to pause the boot process until /dev/block/mmcblk0p2 becomes available? I do not have a dev environment setup - all I have is Cygwin that I use to edit boot.img, port some ROMs to test etc. Can someone help me create such a binary please? Again, I am good at logic and bad at writing/ compiling stuff
Alternatively, can cLK be made to wait until /dev/block/mmcblk0p2 is available? Thoughts?
What is the difference to this methods:
http://forum.xda-developers.com/showthread.php?t=1716124
Are you sure we need more I/O performance with sequencial r/w? Isn't the task with multitasking sutuations to have more perfomance with random r/w? Sdcards won't give that much improvement then: http://www.tomshardware.com/reviews/microsdhc-memory-card-performance,3011-12.html
ReinerK said:
What is the difference to this methods:
http://forum.xda-developers.com/showthread.php?t=1716124
Are you sure we need more I/O performance with sequencial r/w? Isn't the task with multitasking sutuations to have more perfomance with random r/w? Sdcards won't give that much improvement then: http://www.tomshardware.com/reviews/microsdhc-memory-card-performance,3011-12.html
Click to expand...
Click to collapse
In the CronMod as well as all A2SD methods, the Linux boot init process will mount /data to the internal NAND partition (userdata). During Android boot, /data or a part of /data is remounted or symlinked to the SD-EXT partition. This method has its own set of issues, both known and unpredictable. Read up the ROM threads with A2SD or similar stuff in HD2 Android NAND forum and HD2 Android General forum and you will know. This is a trial and error method which is a hit or a miss.
What I am trying to achieve is to have the Linux boot init process mount /data directly to the EXT partition in the SD Card, making this transparent to Android. As far as Android is concerned, it will see a /data partition like it sees a NAND /data partition when you do not use any of these scripts. With this method, there is no miss that I have experienced so far in my testing, except not being able to get the SD Card device ready in time for the Linux boot init process to mount it as /data. Once that is fixed, am sure many will see the benefits.
While the benchmark you have linked to gives a comparison of random and sequential transfer rates of Flash Memory from 9 months ago (it does not even have all that many decent class6 cards that are available now), have you find anything on the transfer rates of NAND? While I remember a HD2 user benchmarking NAND transfer rates, I do not remember the exact values but do remember it was not good. From my personal opinion, the same ROM felt faster with this modification - note that after the first boot all the apps (system or user) that you see and use will be loaded from dalvik-cache which is located in /data. All app data and databases are read/ written in the /data partition. Hence the responsiveness of the phone will be directly proportional on how fast the data partition is - in my case it is my class6 SD Card's EXT partition.
There is also the issue with bad block NAND making several flashaholics' phones unusable or unstable. EXT2/3/4 partitions have better data consistency management and also have e2fsck - it would not be without a reason that Google moved to EXT4 NAND format for their recent phones
Even if you have an International HD2 with 512 MB NAND, with this method you can use almost all of that space (except spaces reserved for bootloader, recovery, misc, etc.) for /system partition which will definitely make the phone live a lot longer even with bad blocks, what with recent cLK versions letting you manipulate partition layouts.
I can keep going on the benefits of having EXT partitions and the benefits of this method, but that would beat the purpose of this thread. Let's see if we can get EXT mounted during boot to /data reliably and consistently. Whether anyone wants to use that method is, of course, their choice
By the way, have anyone used the test ROMs? Any feedback?
I think yaffs file system is used by HTC only, other droid phones already using EXT file system on NAND.
Thats why some phones has good I/O benchmark
Even if it was possible to use EXT4 on HTC HD2 nand would be big improvment
ph03n!x said:
...This is a trial and error method which is a hit or a miss.
Click to expand...
Click to collapse
That's true. Sine even tytung Roms are bigger than 150MB I used 40a2sdx, and I noticed performance problems and sometimes low sound bug. Well I can live with this, but without of cource would be better
ph03n!x said:
While the benchmark you have linked to gives a comparison of random and sequential transfer rates of Flash Memory from 9 months ago
Click to expand...
Click to collapse
I am sorry, I did not found any newer tests in english language. But I think 9 month old test are okay, cause most people use their µSDs several years (my 2009 SanDisk 16GB one just died ).
ph03n!x said:
There is also the issue with bad block NAND making several flashaholics' phones unusable or unstable
Click to expand...
Click to collapse
They should just stop doin' task29 whenever they only wanna flash a rom or change the partition layout.
ph03n!x said:
I can keep going on the benefits of having EXT partitions and the benefits of this method, but that would beat the purpose of this thread. Let's see if we can get EXT mounted during boot to /data reliably and consistently. Whether anyone wants to use that method is, of course, their choice
Click to expand...
Click to collapse
I am pleased to hear that you wanna find a way that way. I just don't have an extra phone to test your rom atm.
Anyway, thaks a lot for your explanations!
ph03n!x said:
In the CronMod as well as all A2SD methods, the Linux boot init process will mount /data to the internal NAND partition (userdata). During Android boot, /data or a part of /data is remounted or symlinked to the SD-EXT partition. This method has its own set of issues, both known and unpredictable. Read up the ROM threads with A2SD or similar stuff in HD2 Android NAND forum and HD2 Android General forum and you will know. This is a trial and error method which is a hit or a miss.
Click to expand...
Click to collapse
Reason I believe it is hit or miss is because during boot init.d scripts in most of the roms runs in parallel. hence while scripts is trying to re-mount/re-link the data partition Android boot-up continues and causes problem. if we pause the android bootup while scripts finishes its job of setting up links correctly I think problem should go away. take a look at this
Code:
# Execute files in /system/etc/init.d before booting
# service sysinit /system/bin/logwrapper /system/xbin/busybox run-parts /system/etc/init.d
# class main
# disabled
# oneshot
if we change that to
Code:
exec /system/bin/sysinit
problem with A2SD scripts should be solved of-course you will have to modify any scripts which sits in infinite loop.
take a look at my latest JB AOSP roms or miui ICS roms.
-----Update----
what if we leave both the method in there
service method to run init.d scripts
and
exec method to pause the boot-up till sd-card is mounted and sd-ext is available.
lets says
make sysinit-1 which will execute the script from init.d1 directory which will have script to check the sd-ext availability
I am close to a solution - looks like a few other devices have already mounted EXT partitions in the SD Card to /system or /data. Here is the command:
Code:
devwait /dev/block/mmcblk0p2
mount ext2 /dev/block/mmcblk0p2 /data nodev nosuid noatime nodiratime
(Check this thread).
But the problem is it is still not mounting. Does this need anything in the kernel/ init? Even the batter-pull technique is not working! The phone is booting as if it is a fresh installation, though I do not see any mounted /data partition
The other option from the same thread is to have
Code:
pause (5);
in init.c and compile is at a binary.
Another bit of information I have come across:
Code:
on emmc-fs
# mount mmc partitions
wait /dev/block/mmcblk0p13
exec /system/bin/e2fsck -p /dev/block/mmcblk0p13
mount ext4 /dev/block/mmcblk0p13 /data nosuid nodev barrier=1 noauto_da_alloc,noinit_itable,nodelalloc
(from here).
With this, the phone is going back to the Encryption Unsuccessful mode.
Pretty sleepy already, gonna call it a day for now - my poor phone has undergone a bunch of boots and reboots for a day
smokin901 said:
Reason I believe it is hit or miss is because during boot init.d scripts in most of the roms runs in parallel. hence while scripts is trying to re-mount/re-link the data partition Android boot-up continues and causes problem. if we pause the android bootup while scripts finishes its job of setting up links correctly I think problem should go away. take a look at this
Code:
# Execute files in /system/etc/init.d before booting
# service sysinit /system/bin/logwrapper /system/xbin/busybox run-parts /system/etc/init.d
# class main
# disabled
# oneshot
if we change that to
Code:
exec /system/bin/sysinit
problem with A2SD scripts should be solved of-course you will have to modify any scripts which sits in infinite loop.
take a look at my latest JB AOSP roms or miui ICS roms.
Click to expand...
Click to collapse
Thanks, I will check your ROM. I would still like to get /data to sd-ext working before Android boots though I thought I was close today, but unfortunately no!
Looks like there had been some progress to this cause, thanks to @securecrt! Check this post.
Swyped from my HTC HD2 using XDA Premium
ph03n!x said:
Mounting - /dev/block/mmcblk0p2 is not getting mounted to /data at every boot/ reboot reliably. This results is the phone not booting at all (Gingerbread) or giving an Encryption Failed, wipe data prompt (This is a feature of ICS/ JB, if /data is not accessible, the OS will boot with a temporary /data file system to show this error).
I have not been able to find a way to reliably carry out this mount - my best guess is that the SD Card Reader interface is not getting ready in time to be mounted to /data. I do not know how to make the boot process wait until the device is available.
ClockWorkMod/ Other recoveries - As the recovery boot images are still mounting userdata to /data and the ext partition to sd-ext, you will have to access sd-ext to access your data. What this means is that if you are moving ROMs, a direct CWM backup restore won't work unless the new ROM also has this modification
Performance- is only as good or bad as your SD Card. My second HD2 has a class 2 SD Card, and I do not find any performance difference. While this is not bad, the SD Card I am talking about is a decent SanDisk card. If you have something with poor transfer rates, performance will suffer
SD Card cannot be removed from the system while Android is running!!!
Click to expand...
Click to collapse
This is, obviously, not compatible with other setups, such as those using some type of ad2sd script (i.e. Amarulls), correct?
CWM Recovery backup should still work since it does backup sd-ext, no?
I need another sd card to play..
Ok, I need some help-
securecrt said:
the trick is:
1. rename the init to init.android
2. create a script named init, mount the system/data in this script, you can add a few sleep time in this to confirm the mount is alreay done and then execut init.android
3. disable the system/data mount in init.rc
Click to expand...
Click to collapse
Ever since @securecrt posted this, I have been running in circles. Here is what I have done:
Renamed init to init.android
Commented out in init.rc:
Code:
#mkdir /data 0771 system system
Commented out in init.rc:
Code:
#mount ext4 /dev/block/mmcblk0p2 /data nosuid nodev noatime nodiratime
Created a script named init with
Code:
#!/sbin/busybox sh
export PATH=/sbin
sleep 10
mkdir /dev
mkdir /dev/block
mknod /dev/block/mmcblk0p2 b 179 2
mkdir /data 0771 system system
mount ext4 /dev/block/mmcblk0p2 /data nosuid nodev noatime nodiratime
/init.android
Copied busybox binary to /sbin/busybox
Created the initrd.gz and flashed
The phone is booting to second boot (logo.rle loads), and then reboots.
Like I've said I have very limited Linux/ coding skills. Would be great if someone can nudge me in the right direction!
ny_limited said:
This is, obviously, not compatible with other setups, such as those using some type of ad2sd script (i.e. Amarulls), correct?
CWM Recovery backup should still work since it does backup sd-ext, no?
Click to expand...
Click to collapse
Yes, you need to remove all variants of a2sd/ link2sd scripts.
CWM Recovery will back up SD-EXT as /sd-ext, but Android will load it as /data after this mod. So if you are going to try this mod, you need to titanium backup all you system/ user app+data, install the modified mod or initrd.gz, make sure you have a ext partition, boot to Android and restore the backup from titanium. And the exact same if you are removing the mod (in which case it will be original initrd.gz).
I can sense that I am pretty close to getting this to work reliably, except for being a dunce in Linux/ Scripting. With little help this mod is a go for daily use
ph03n!x said:
Ok, I need some help-Ever since @securecrt posted this, I have been running in circles. Here is what I have done:
Renamed init to init.android
Commented out in init.rc:
Code:
#mkdir /data 0771 system system
Commented out in init.rc:
Code:
#mount ext4 /dev/block/mmcblk0p2 /data nosuid nodev noatime nodiratime
Created a script named init with
Code:
#!/sbin/busybox sh
export PATH=/sbin
sleep 10
mkdir /dev
mkdir /dev/block
mknod /dev/block/mmcblk0p2 b 179 2
mkdir /data 0771 system system
mount ext4 /dev/block/mmcblk0p2 /data nosuid nodev noatime nodiratime
/init.android
Copied busybox binary to /sbin/busybox
Created the initrd.gz and flashed
The phone is booting to second boot (logo.rle loads), and then reboots.
Like I've said I have very limited Linux/ coding skills. Would be great if someone can nudge me in the right direction!
Click to expand...
Click to collapse
I am sure securecrt will reveal his secret soon, m8.. I wouldn't 'waste' my time on it since he is already running it and knows how to do it
Maybe you can send him a PM if the wait is killing you ?

How to put /data on external SD

WORD OF WARNING: if you do this, wiping /data from recovery will not actually wipe data. AND if you install a new ROM, via clean or dirty flash, chances are your large /data partition will go all screwy on you. It did for me. If I flashed anything, it tends to cause boot loops or hangs at the boot animation.
If you are dirty flashing anything, I would first get as much off of the large /data as you can to get it to 2 GB or less, then drop it on top of your 2 GB /data partition, then dirty flash, let everything come back up and get stable then repeat these steps starting at step 2 to blow away your large /data and then mirror your 2 GB /data to your large /data.
The problems I had could be related to xposed, but I am not 100% certain on this. It is my best guess from looking at my logcat (doesn't actually indicate a crash in there, but has a lot of stuff related to xposed).
These steps are more for the non-flashaholics amung us OR people who flash new stuff but roll back to nandroid. Once you pop the larger /data onto any ROM, your nandroid will likely not play nice with it anymore as your larger /data is not restored from a nandroid.
And the normal "warranty is now void" and "if things break, it isn't my fault" disclamer stuff too. But I'm running this right now and liking it (except when I go to jump to KK it may not be happy with me... gonna need to do some testing to nandroids work properly AND so I can flash ROMs without blowing everything up... thinking it is just the dalvik that is unhappy, but I am uncertain at this time)
I was having issues with my /data partition getting full on me after getting new ROMs set up. This was irritating me and I came to the conclusion that 2 GB was just not enough space for some of the larger apps out there. So I went ahead and started investigating and testing how to get /data onto the external SD card.
2 complications arise:
1) it needs to be in ext4 format to preserve permissions
2) it needs to be swapped post boot AFTER the sd card is mounted (potentially, I have not fully tested this).
So this becomes a multi step process. What you need:
1) an external SD card that is at least 2 GB in size if not larger (I recommend larger simply because what is the point of having it on external storage if you aren't gaining any extra space)
2) a SGH-T989D (I do not have an SGH-T989 to test it with BUT I am confident it will work with that one too)
3) some partitioning software on your PC (I recommend EaseUS or AOMEI, but use whatever you are comforatble with)
4) latest version of busybox installed on your phone
So, the steps to do this:
1) partition your external SD card into at least 2 partitions, where your second partition will be used for /data. The first partition automatically mounts to /storage/sdcard1, but partition 2 will NOT auto-mount. You do not need to format this if you do not want to. This is up to you how you do it.
2) Run the following commands from either adb shell OR android terminal:
su
mke2fs /dev/block/mmcblk1p2
mkdir /storage/sdcard1/PARTITION2
mount -t ext4 /dev/block/mmcblk1p2 /storage/sdcard1/PARTITION2
rsync -HpogEvrl /data/* /storage/sdcard1/PARTITION2
busybox umount -l /storage/sdcard1/PARTITION2
mount -o remount,errors=continue /data
busybox umount -l /data && mount -t ext4 -o rw,seclabel,errors=continue,user_xattr,acl,barrier=1,data=ordered,noauto_da_alloc /dev/block/mmcblk1p2 /data
chown system:system /data
restorecon /data
brief explaination of the above commands:
su - switch user to root
mke2fs - format the block device as ext4
mkdir - make directory
mount -t ext4 - mount a block device of type ext4 from /dev/block/mmcblk1p2 to /storage/sdcard1/PARTITION2
rsync -HpogEvrl - copy files from /data/* to /storage/sdcard1/PARTITION2 preserving all attributes (including owner and group)
busybox umount -l - use the busybox version of umount to do a lazy unmount of /storage/sdcard1/PARTITION2. this will unmount it even if files are locking it
mount -o remount,errors=continue /data - remounts the data partition telling it to continue instead of kernel panic if there is any errors reading or writing to that mount point
and the next command is actually 2:
busybox umount -l /data && mount -t ext4 -o rw,seclabel,errors=continue,user_xattr,acl,barrier=1,data=ordered,noauto_da_alloc /dev/block/mmcblk1p2 /data
this does a lazy unmount of /data and then mounts /dev/block/mmcblk1p2 to /data with the appropriate parameters for the /data parameter
chown system:system /data - This gives system ownership of /data instead of root as is originally done
restorecon /data - we do this just in case the userdata partition has been reset.
3) OPTIONAL delete the directory /storage/sdcard1/PARTITION2. This is just cleanup, it is not neccessary to do.
When you reboot your phone, your /data directory will remount to the 2 GB partition as it did originally. To remount it to your newly created /data partition, run the following 2 commands:
su
busybox umount -l /data && mount -t ext4 -o rw,seclabel,errors=continue,user_xattr,acl,barrier=1,data=ordered,noauto_da_alloc /dev/block/mmcblk1p2 /data
Now, you can make this into a nice and easy startup script by following these simple steps from adb shell or android terminal (NOTE this assumes that your ROM supports userinit.d and init.d):
su
cd /data/local/userinit.d
echo "#!/system/bin/bash" > dataswap
echo "if [ -e \"/dev/block/mmcblk1p2\" ]" >>dataswap
echo "then" >> dataswap
echo "busybox umount -l /data && mount -t ext4 -o rw,seclabel,errors=continue,user_xattr,acl,barrier=1,data=ordered,noauto_da_alloc /dev/block/mmcblk1p2 /data" >> dataswap
echo "chown system:system /data" >> dataswap
echo "restorecon /data" >> dataswap
echo "fi" >>dataswap
chown root:shell dataswap
chmod 777 dataswap
and now when you reboot your phone you will have your external SD cards partition 2 as your /data directory.
If your phone has init.d support, look in your /system/etc/init.d folder for a script named "90userinit". If it is not there, you will need to either make it and have it run userinit.d OR change the line:
cd /data/local/userinit.d
to
cd /system/etc/init.d
mount -o remount,rw /system
now for the warning stuff:
1) I strongly recommend that you do NOT put this into your /data/local/userinit.d folder until you are 110% certain that it worked on your device with your ROM. The reason for this is if it fails and you don't have it in that folder, simply reboot and your phone is back to using /data from partition 25 on your internal memory
2) I strongly recommend you do a nandroid before you begin. That way in the very slim chance that something goes horridly wrong, you can recover
3) I strongly recommend you back up your external SD card before you begin resizing your partitions. Resizing partitions may result in data loss.
4)I take no responsibility for any data loss
5) I have ONLY tested this on a SGH-T989D running HellyBean (Android 4.3). I cannot make any comments on other ROMS, but as long as the ROM has busybox, root, and init.d support, this should work.
6) If you get ANY ERRORS from ANY of the above commands, stop everything and either post in this thread or start over. I have done this twice on my phone to make sure that the steps are as solid as I know how to make them. The only thing I haven't fully tested is putting it into a startup script.
One thing I notice, sometimes when you unmount and remount /data, any data changes in your currently running apps (ie you get a text message while doing this or you start a game up while /data is being sync'ed) may cause odd data issues when the two swap and may cause your phone to jump back to the boot animation. If this occurs, try waiting approximately 1 minute OR until adb shell drops out; whatever comes first. At that point, something has likely gone sour. Reboot your phone and things will be back how they were prior to step 2 (step 1 is permanent until you change that).
Questions, comments, suggestions? Please leave them in the thread, not in PM. If I get any PM's about this thread, I'll do my best to answer, but I may end up posting your PM in this thread to help others.
EDIT: cleaned up the instructions a little and added in some recommended steps (the chown and restorecon) and removed the waiting to mount /data as that just makes things unhappy long term.
EDIT: added error handling in case mmcblk1p2 does not exist to the startup script. If it does not exist, it will now NOT swap the partitions. So as a failsafe, if your phone gets stuck at the boot animation, you can power it down, remove the SD card and power it back up to boot with your default /data partition. This should not need to be done, but there is a chance that your SD card could die or your new /data partition get corrupted, this will allow you to recover safely and will likely only require you to re-create your data partition and NOT have to re-do your whole ROM due to something going bad on the big /data.
Dang...found a bug: it looks like if you leave it for a few days or reboot our something(have not narrowed it down yet) some apps stop working... investigating it and should have a solution sometime tonight... sorry to all those who have tried this... if you have not done much, I'd suggest switching back to normal /data until I get the bug sorted out. My best guess is that reboot doesn't unmount the partition cleanly but I'm not positive...
bmg002 said:
I was having issues with my /data partition getting full on me after getting new ROMs set up. This was irritating me and I came to the conclusion that 2 GB was just not enough space for some of the larger apps out there. So I went ahead and started investigating and testing how to get /data onto the external SD card.
2 complications arise:
1) it needs to be in ext4 format to preserve permissions
2) it needs to be swapped post boot AFTER the sd card is mounted (potentially, I have not fully tested this).
So this becomes a multi step process. What you need:
1) an external SD card that is at least 2 GB in size if not larger (I recommend larger simply because what is the point of having it on external storage if you aren't gaining any extra space)
2) a SGH-T989D (I do not have an SGH-T989 to test it with BUT I am confident it will work with that one too)
3) some partitioning software on your PC (I recommend EaseUS or AOMEI, but use whatever you are comforatble with)
4) latest version of busybox installed on your phone
So, the steps to do this:
1) partition your external SD card into at least 2 partitions, where your second partition will be used for /data. The first partition automatically mounts to /storage/sdcard1, but partition 2 will NOT auto-mount. You do not need to format this if you do not want to. This is up to you how you do it.
2) Run the following commands from either adb shell OR android terminal:
su
mke2fs /dev/block/mmcblk1p2
mkdir /storage/sdcard1/PARTITION2
mount -t ext4 /dev/block/mmcblk1p2 /storage/sdcard1/PARTITION2
rsync -HpogEvrl /data/* /storage/sdcard1/PARTITION2
busybox umount -l /storage/sdcard1/PARTITION2
mount -o remount,errors=continue /data
busybox umount -l /data && mount -t ext4 -o rw,seclabel,errors=continue,user_xattr,acl,barrier=1,data=ordered,noauto_da_alloc /dev/block/mmcblk1p2 /data
brief explaination of the above commands:
su - switch user to root
mke2fs - format the block device as ext4
mkdir - make directory
mount -t ext4 - mount a block device of type ext4 from /dev/block/mmcblk1p2 to /storage/sdcard1/PARTITION2
rsync -HpogEvrl - copy files from /data/* to /storage/sdcard1/PARTITION2 preserving all attributes (including owner and group)
busybox umount -l - use the busybox version of umount to do a lazy unmount of /storage/sdcard1/PARTITION2. this will unmount it even if files are locking it
mount -o remount,errors=continue /data - remounts the data partition telling it to continue instead of kernel panic if there is any errors reading or writing to that mount point
and the last command is actually 2:
busybox umount -l /data && mount -t ext4 -o rw,seclabel,errors=continue,user_xattr,acl,barrier=1,data=ordered,noauto_da_alloc /dev/block/mmcblk1p2 /data
this does a lazy unmount of /data and then mounts /dev/block/mmcblk1p2 to /data with the appropriate parameters for the /data parameter
3) OPTIONAL delete the directory /storage/sdcard1/PARTITION2. This is just cleanup, it is not neccessary to do.
When you reboot your phone, your /data directory will remount to the 2 GB partition as it did originally. To remount it to your newly created /data partition, run the following 2 commands:
su
busybox umount -l /data && mount -t ext4 -o rw,seclabel,errors=continue,user_xattr,acl,barrier=1,data=ordered,noauto_da_alloc /dev/block/mmcblk1p2 /data
Now, you can make this into a nice and easy startup script by following these simple steps from adb shell or android terminal (NOTE this assumes that your ROM supports userinit.d and init.d):
su
cd /data/local/userinit.d
echo "#/sys/bin/bash" > dataswap.sh
echo "while [ \`mount | grep -c sdcard\` -lt 3 ]" >> dataswap.sh
echo "do" >> dataswap.sh
echo "sleep 10" >> dataswap.sh
echo "done" >> dataswap.sh
echo "busybox umount -l /data && mount -t ext4 -o rw,seclabel,errors=continue,user_xattr,acl,barrier=1,data=ordered,noauto_da_alloc /dev/block/mmcblk1p2 /data" >> dataswap.sh
chown root:shell dataswap.sh
chmod 777 dataswap.sh
and now when you reboot your phone you will have your external SD cards partition 2 as your /data directory.
If your phone has init.d support, look in your /system/etc/init.d folder for a script named "90userinit". If it is not there, you will need to either make it and have it run userinit.d OR change the line:
cd /data/local/userinit.d
to
cd /system/etc/init.d
mount -o remount,rw /system
now for the warning stuff:
1) I strongly recommend that you do NOT put this into your /data/local/userinit.d folder until you are 110% certain that it worked on your device with your ROM. The reason for this is if it fails and you don't have it in that folder, simply reboot and your phone is back to using /data from partition 25 on your internal memory
2) I strongly recommend you do a nandroid before you begin. That way in the very slim chance that something goes horridly wrong, you can recover
3) I strongly recommend you back up your external SD card before you begin resizing your partitions. Resizing partitions may result in data loss.
4)I take no responsibility for any data loss
5) I have ONLY tested this on a SGH-T989D running HellyBean (Android 4.3). I cannot make any comments on other ROMS, but as long as the ROM has busybox, root, and init.d support, this should work.
6) If you get ANY ERRORS from ANY of the above commands, stop everything and either post in this thread or start over. I have done this twice on my phone to make sure that the steps are as solid as I know how to make them. The only thing I haven't fully tested is putting it into a startup script.
One thing I notice, sometimes when you unmount and remount /data, any data changes in your currently running apps (ie you get a text message while doing this or you start a game up while /data is being sync'ed) may cause odd data issues when the two swap and may cause your phone to jump back to the boot animation. If this occurs, try waiting approximately 1 minute OR until adb shell drops out; whatever comes first. At that point, something has likely gone sour. Reboot your phone and things will be back how they were prior to step 2 (step 1 is permanent until you change that).
Questions, comments, suggestions? Please leave them in the thread, not in PM. If I get any PM's about this thread, I'll do my best to answer, but I may end up posting your PM in this thread to help others.
Click to expand...
Click to collapse
So found a solution to crashing apps . Load up the play store and update your apps. Worked for me anyways.
Now to fix a potential mounting bug in the op, put any scripts on a different ext4 position such as/system/xbin or/system/etc/init.d.
I'll keep you posted if I find other bugs.
Sent from my SAMSUNG-SGH-T989 using xda app-developers app
Bleh new bug:
When you unplug it from a pc, it gets confused and you get io errors which cause it to crash back to the boot animation and get stuck there... too tired to debug out tonight but I'll poke at it tomorrow... thinking I may need to build a kernel module for this... or a custom kernel
Sent from my SAMSUNG-SGH-T989 using xda app-developers app
So the unplugging issue seems quite intermittent and I am unable to get consistent results with it. So I am not sure if it is a bug or not. I updated the OP to have error handling so you can pull your SD card out to mount the 2 GB /data that your phone normally does instead of your big one in the event that something goes wrong.
Hrm... so my testing and further development with this has taken a temporary pause due to my SD card dying on me. Stupid SD card... haven't even had it a year. Hoping I can find the reciept. But once I get it replaced, I'll be jumping in on this some more. I have a few cool ideas that we can do by moving partitions to external storage...
What's the deal with cause this would so help with games but is it possible to maybe if possible extend the internal storage
snt? from? sumthiN 4.4+ ?via a ?T989
hatememarkz said:
What's the deal with cause this would so help with games but is it possible to maybe if possible extend the internal storage
snt? from? sumthiN 4.4+ ?via a ?T989
Click to expand...
Click to collapse
From my testing it seems to work but puts a heavy load the sd card. And if your sd card fails you will lose some data.
I have not tested this on KitKat yet but that is my plan soon.
Sent from my SM-T310 using xda app-developers app

mounts2sd - increase internal storage

I have saw some requests here for increasing internal storage and only few solutions.
Because needed to use similar tool in the past on mini defy with only 512MB ROM I've tried the same tool - Mounts2sd - also here on C1905 running cm11.
All what was needed was prepared microSD, init.d script which mounts second ext4 partition and Mounts2sd application installed from Google play store (but available also here in XDA forum on application home page)
For testing it was used 4GB microSD card with first 1.9GB vfat partition and second 1.8GB ext4 formatted partition, m2sd was configured to use cm11 busybox and to move applications to SD card.
Mount sd-ext script was copied to /system/etc/init.d directory with permission 755.
Code:
#!/system/bin/sh
blkdev=/dev/block/mmcblk1p2
mntpoint=/sd-ext
mntpoint2=/mnt/media_rw/sdcard2
mntpoint3=/storage/sdcard2
# otherwise m2sd complains: The system partition is S-On protected...
mount -o remount,rw /
mount -o remount,rw /system
if [ ! -d $mntpoint ]; then
mkdir $mntpoint
ln -s $mntpoint $mntpoint2
ln -s $mntpoint $mntpoint3
fi
mount -t ext4 -o rw,errors=remount-ro $blkdev $mntpoint
unset mntpoint
unset blkdev
unset mntpoint mntpoint2 mntpoint3
exit 0
You can use script also self standing on rooted phone and run it every time you would like to have second partition available.
Script to survive CM update.
Thanks brother, nice guide, on my list for weekend.
---------- Post added at 12:43 AM ---------- Previous post was at 12:40 AM ----------
I think im done with thanks quota for today

Categories

Resources