Develop Bugs

[HOWTO][KERNEL][G800F][exynos] Custom kernel tutorial

This is a short tutorial on how to compile and flash a (customized) Linux kernel on your G800F.
Note that this tutorial is only for the G800F as G800H/M/Y/... have a totally different architecture (SoC,...).
Toolchain
First of all get a decent toolchain (gcc,...). There are some prebuilt ones at android.googlesource.com.
If you have git installed fetch the arm-eabi-4.6 toolchain with:
Code:
git clone https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/arm/arm-eabi-4.6
Other toolchains might work but I had no luck with a Linaro toolchain optimized for the Cortex-A7 (ARM-architecture of the Exynos 3470) - the kernel did not boot at all.
Also the arm-eabi-4.8 toolchain from android.googlesource.com aborted with a compile error, so use the 4.6 version.
Fetch the Linux Kernel sources
Now you have to fetch the kernel sources. At the moment there are only the official sources from Samsung's open source server.
http://opensource.samsung.com/reception/receptionSub.do?method=sub&sub=F&searchValue=SM-G800F
Select a source package:
The source packages at this server contain the kernel sources (Kernel.tar.gz), some part of the Android system (Platform.tar.gz) and some build instructions (README_*). There are multiple versions of the source packages. You can find an explanation of the versioning scheme here: samsung-firmware-version-number.html
For example the version part of a source package G800FXXU1ANG1 is NG1 which stands for N=2014, G=July, 1=1st release.
G800FXXU1ANJ2 is N=2014, J=October, 2nd release.
In addition Samsung adds country code post-fixes like "_MEA" which might contain regional adaptions. See http://forum.xda-developers.com/showpost.php?p=57410625&postcount=5 for an explanation and list of country codes. For example MEA probably stands for "MIDDLE EAST ASIA".
So the newest source packages are G800FXXU1ANG1 and G800FXXU1ANG4 (MEA). I compared the kernel sources of both packages and found no difference. Seems as if only the Android part was changed. Both are from July so not very up-to-date. At least they are based on a 3.4.39 kernel which matches the kernel version of G800FXXU1ANL1.
Mirror:
Note that downloading a source package from Samsung's Open Source server takes a lot of time as the download speed is very poor. It took 12-24 hours for me.
You can also download the sources from my github repository:
https://github.com/tobigun/samsung-kernel-smg800f
At the moment the master contains the G800FXXU1ANG1 kernel sources but might be updated as soon as a newer kernel source is released.
Compile the Kernel
Prepare the build by applying the S5 Mini kernel config:
Code:
ARCH=arm CROSS_COMPILE=<PATH_TO_TOOLCHAIN>/bin/arm-eabi- make kminilte_00_defconfig
If you want to change the config now, you can do this with:
Code:
ARCH=arm CROSS_COMPILE=<PATH_TO_TOOLCHAIN>/bin/arm-eabi- make menuconfig
After you have done some changes to the kernel sources you can build it with:
Code:
ARCH=arm CROSS_COMPILE=<PATH_TO_TOOLCHAIN>/bin/arm-eabi- make -j4
If you do not want to build with multiple CPUs you can remove the "-j4" from the command line (which also is handy if an error occurred).
In either case the resulting kernel image will be in "arch/arm/boot/zImage".
Note that kernel module support is disabled in the S5 mini config so everything you need (all drivers, ...) is in the zImage. There is no need to copy additional kernel modules to the filesystem.
Flashing the Kernel
Almost done. It is not possible to flash the kernel image directly. Instead you have to put the kernel image and a ramdisk together into a flasable boot.img. The easiest way to get such a boot.img is to use an existing boot.img and replace just the kernel image.
1. Get a decent boot.img
There are at least two ways:
a) Download the current firmware image from SamMobile (http://www.sammobile.com/firmwares/database/SM-G800F/) and extract the boot.img. With 7-zip there will be an error message if you extract the .tar.md5 inside the zip - just ignore the message and extract the boot.img file.
b) Extract the current boot.img from the device's flash with (root required):
Connect to the device (with "adb shell" or ssh).
Then copy the contents of the boot-partition to a file:
Code:
dd if=/dev/block/mmcblk0p9 of=boot.img
2. Replace kernel image
For this step AIK (Android Image Kitchen) is required.
Download it at http://forum.xda-developers.com/showthread.php?t=2073775. Probably you want AIK-Linux-v1.7-ALL.tar.gz (or newer).
Extract the files.
Now unpack the boot image, replace the kernel image and repack it (with the original ramdisk) to a new boot.img:
Code:
./unpackimg.sh boot.img
cp <KERNEL_BUILD_DIR>/arch/arm/boot/zImage split_img/boot.img-zImage
./repackimg.sh --original
Note that the zImage name inside split_img might differ from boot.img-zImage.
The new boot.img is "image-new.img". Rename it to boot.img and flash it.
3. Flash new boot.img
Important notice:
- You flash this image at your own responsibility. I am not responsible for any damage that might be caused by flashing this image (bricked device, lost data, ...)
- Flashing this kernel image will trigger the KNOX counter, so your warranty will be void.
- The image is only for S5 Mini SM-G800F (Exynos)
- The kernel might be instable, crash your device, drain your battery, or even might damage your smartphone
- Backup your data before flashing
- Make sure you have the current firmware of your device (e.g. from SamMobile) so you can flash it back in case something goes wrong.
There are multiple ways to flash the boot.img to your device:
a) From recovery with an update.zip
b) Using Odin
c) Directly with dd
a) From recovery with an update.zip
TODO
b) Using Odin
Create tar.md5:
At least some versions of Odin3 (e.g. v3.07) need a tar.md5 instead of a "simple" tar file. Although a simple tar-file can be selected in Odin, the image will not be flashed correctly.
Download tar-Tool_Odin3-v3.07_by_mkh.mourad.zip
Extract it.
On windows copy the boot.img file into the tar-Tool folder first and then drag and drop the boot.img on ImgToTar.MD5.bat. Alternatively you can also pass the filename as a command line parameter to ImgToTar.MD5.bat.
Now you should have a boot.tar.md5.
Flash tar.md5:
Download Odin (for example Odin3-v3.07 contained in CF-Auto-Root)
Reboot your device into Odin mode: turn off your device, then press Volume-Down + Home + Power button at the same time and release them. Confirm the following message with the Volume-Up button.
Connect your device to your PC via USB
Make sure the device driver's are installed on your PC
Start Odin
Select PDA and select the kernel image (boot.tar.md5)
Check that only "Auto Reboot" and "F. Reset Time" is set
Click on "Start": the kernel image should be flashed now and the device should reboot afterwards
c) Directly with dd
Connect to the device (with "adb shell" or ssh).
Then copy the contents of the boot.img file to the boot-partition:
Code:
dd if=boot.img of=/dev/block/mmcblk0p9
Reboot
Done
You can now check in "Settings - Device Info - Kernel-Version" if your kernel is used.

Reserved

[Kernel][Exynos][BROKEN] Kali NetHunter for the Galaxy S7

OKAY SO
Currently, I am unable to get even stock kernel sources to boot. I'm not sure what to do to at this point. Waiting on someone else that actually has the device to get this figured out - DO NOT FLASH!
WARNING: This is completely untested, highly theoretical, and possibly dangerous. Flash at your own risk.
Back up your original boot image in TWRP before attempting to flash this! If it doesn't boot, you can simply restore your previous boot image.
This is Kali NetHunter 3.0.5 for the Galaxy S7.
If you don't know what Kali NetHunter is, well, it's the entire Kali Linux operating system in a chroot on your phone, plus a bunch of awesome apps for executing exploits, fixing things, doing cool things. It goes on, I suppose.
I'm gonna be honest guys, I'm not a security person. When it comes to security, I'm more of a Paul Blart.
What I do know though, is that there is apt-get, and apt-get is life.
Find much more information here: https://github.com/offensive-security/kali-nethunter/wiki
The answer to all your questions, generally the answer is YES, IT CAN DO THAT.
Most ROMs should be supported, as our installer uses a dynamic patching method on your current boot image!
The updater zip will add a few files to your /system partition, and install all of the NetHunter apps to your /data partition.
The chroot is located in /data/local, so you don't have to worry about your system partition being full. It's full read/write capable.
Understand that the zip will replace your current kernel with a completely different one.
This is necessary because most stock or custom kernels don't provide the drivers needed to operate most of Kali NetHunter's features.
DOWNLOAD
Current version: 3.0.5 (beta, 2016-03-11)
Please be careful to download the right version based on this table:
SM-G930F, SM-G930FD, SM-G930X, SM-G930W8: herolte
SM-G935F, SM-G935FD, SM-G935X, SM-G935W8: See proper forum.
All others be sad.
Download is available at: https://idlekernel.com/nethunter/herolte/
Grab the 700 MB+ zip.
Kernel-only zip is for upgrading your kernel, or just using the NetHunter kernel by itself. (yes, you can do that!)
BEFORE INSTALLING
IMPORTANT: Kali NetHunter requires write access to your data partition!
Flash this zip in TWRP to allow system modifications and unencrypted data: https://idlekernel.com/fun-stuff-trust-me/no-verity-opt-encrypt.zip
Once that is flashed, go to the wipe page and use the [Format Data] button.
This will wipe all your data, including internal storage!
Boot up your system and set up Android.
Now you can go back into TWRP and flash Kali NetHunter.
Currently, Samsung encryption is not supported by TWRP, so we have to disable it.
Sorry security freaks! There's a lot of irony here, isn't there?
The Kali chroot and apps are installed on your data partition (in /data/local for chroot). To initialize the chroot and install Kali Linux, you need to start the Kali NetHunter app.
The NetHunter installer will automatically install SuperSU (2.68) in system mode, which I consider to be more stable. Since NetHunter already modifies your system partition, there is no need to use systemless SuperSU anyways.
Also included is an extra Busybox that gives you full large file support and some extra applets.
FULL FRESH INSTALL STEPS
Install Team Win Recovery Project to your recovery partition.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
If your data partition doesn't mount in TWRP:
Go to [Wipe] -> [Format Data] (not advanced wipe) -> type "yes".
WARNING: This will wipe your internal storage, disable encryption, and factory reset your phone!
Once your data partition is formatted, go to [Reboot] -> [Recovery].
Download dm-verity and force encryption disabler.
Without exiting TWRP, transfer no-verity-opt-encrypt.zip to your device over MTP* and flash it using [Install] in TWRP.
If you wiped your data partition in step 2:
Go to [Reboot] -> [System].
Set up your phone by following the Android setup wizard.
Once it's set up, reboot back into recovery.
Download Kali NetHunter.
Without exiting TWRP, transfer the NetHunter installer zip to your device over MTP* and flash it using [Install] in TWRP.
Go to [Reboot] -> [System].
Wait 5-15 minutes for your device to finish setting itself up.
Open the NetHunter app to initialize the environment
You're done!
* MTP, known as Media Transfer Protocol, is the same way you transfer files from your PC to your device when booted into system.
UPDATING TO A NEWER BUILD
Going from 3.0.0 and up, all you have to do is flash the new build in recovery and wipe dalvik cache.
UPDATING YOUR ROM
To get all your NetHunter and SuperSU functions back after flashing a new ROM, just flash the ~60 MB update-nethunter-* zip again.
THE KERNEL
The NetHunter kernel for the Galaxy S7 is based on Samsung's OSRC G930FXXU1APAW kernel sources.
It has the following changes:
F2FS updated to Jaeguek Kim's latest kernel.org f2fs-stable sources
F2FS formatted system, data, and cache partition support
UKSM Ultra Kernel Same-page Merging KSM support
Updated and enabled USB (OTG) Atheros, Ralink, and Realtek WiFi drivers
Simple IO (SIO) scheduler as default IO scheduler
USB HID Gadget keyboard support
mac80211 packet injection support
DriveDroid compatibility
Additional drivers built in for the full Kali NetHunter experience
Data partition encryption changed from forced to optional (disabled during installation)
Disables dm-verity and allows you to boot modified system partitions
RAN INTO AN ISSUE OR BUG?
In order for me to help you, you have to at minimum reply with:
The link to the exact zip you downloaded
Your device model (it better not be something other than G900F, dangit!)
The name of the ROM you're flashing it on
The version and build date of the ROM you're flashing it on
A complete description of your problem
Optional: An audio recording of you reading this entire post
If your issue is with a specific app, it might be better to contact the developer of that app.
If your issue is during the installation (ex. flashing the NetHunter zip), then please collect a TWRP recovery.log for me.
If you found a problem and were able to fix it, and no one's mentioned it in the thread already, it would be kind to state the issue and your fix for others to make use of as well.
You can join me and the other NetHunter developers on IRC at the #nethunter room on freenode to more handily diagnose problems together.
I apologize, but I can't do house calls at this time.
KNOWN ISSUES
USB Keyboard - The keyboard is unusable when using Google Keyboard as your input method. Switch to Hacker's Keyboard.
NetHunter Terminal - It doesn't automatically set the columns/rows, so you need to type "resize" sometimes to fix the display.
DEVELOPMENT
You can see my branch of the installer development here: https://github.com/jcadduono/kali-nethunter
Alternatively, the main branch is also available on the Offensive Security GitHub: https://github.com/offensive-security/kali-nethunter
Kernel source: https://github.com/jcadduono/nethunter_kernel_herolte
SCREENSHOTS
DISCLAIMER
I am not affiliated with Offensive Security. They seem like cool guys though.
I'm not even a novice when it comes to security and penetration. I'm just a simple system administrator with a passion for breaking Android.
Please restrain yourselves from asking me security related questions.
XDA:DevDB Information
Kali NetHunter for the Galaxy S7, Kernel for the Samsung Galaxy S7
Contributors
jcadduono, The Kali NetHunter team
Source Code: https://github.com/offensive-security/kali-nethunter
Kernel Special Features:
Version Information
Status: Beta
Current Beta Version: 3.0.5
Beta Release Date: 2016-03-11
Created 2016-03-11
Last Updated 2016-04-15

Awesome! I asked if this was possible years ago. Glad someone is actually making it happen!

phone is rebooting now and then after flashing the kernel.. any idea why?

[email protected] said:
phone is rebooting now and then after flashing the kernel.. any idea why?
Click to expand...
Click to collapse
no you will have to wait for someone to try it that can actually provide useful information such as logs

jcadduono said:
no you will have to wait for someone to try it that can actually provide useful information such as logs
Click to expand...
Click to collapse
Not sure how to get logs when the device doesn't boot could you give us a hint so we can provide you with what you need?

finally, did great work there, but how do i download it
ok did manage it some how the IDM was the issue and cant download it

hm cant flash ur kernel and stuck in boot loop, when i try to flash it its says in twrp "unable to mount /data as rw!" any idea?

SELinux ins still enforcing or do you managed to change it?

Activadee said:
SELinux ins still enforcing or do you managed to change it?
Click to expand...
Click to collapse
The new kernel is selinux permissive if you don't mind trying it for me. It would really help if someone could join #nethunter on freenode IRC to get this working with me.
I've removed the full installer until we can get the kernel working without issues.

I installed now your test 2 Kernel zip. But its causing Bootloop - its dont go over the "Samsung Galaxy S7" logo. Its a European s7 with Exynos. If you tell me HOW to provide some logs i would like to help you if i can ;D
Edit// I Managed to upload the log from booting with your kernel. I don't know if it's right but maybe it helps you. I backed up my boot with twrp, flashed your kernel, reboot until the loop and restored the old boot with twrp
https://drive.google.com/file/d/0B9IHgLrX7UgTRThJUEs3RExUR1k/view?usp=docslist_api

Activadee said:
I installed now your test 2 Kernel zip. But its causing Bootloop - its dont go over the "Samsung Galaxy S7" logo. Its a European s7 with Exynos. If you tell me HOW to provide some logs i would like to help you if i can ;D
Edit// I Managed to upload the log from booting with your kernel. I don't know if it's right but maybe it helps you. I backed up my boot with twrp, flashed your kernel, reboot until the loop and restored the old boot with twrp
https://drive.google.com/file/d/0B9IHgLrX7UgTRThJUEs3RExUR1k/view?usp=docslist_api
Click to expand...
Click to collapse
nope need /sys/fs/pstore not recovery.log
i also need a backup of your boot partition after flashing the kernel

I upload the boot partition later. Would you please tell me how to grab the log from /sys/fs/pstore ? Via Adb from twrp?
// Edit
The files from the boot partition are uploaded
https://drive.google.com/folderview?id=0B9IHgLrX7UgTRnlvcWFiRHVwYUE
I just flashed your Kernel and backed up the partition after this. Or do you need one with already tried to boot?

Activadee said:
I upload the boot partition later. Would you please tell me how to grab the log from /sys/fs/pstore ? Via Adb from twrp?
Click to expand...
Click to collapse
yes it should exist for a few minutes after exiting boot loop and entering twrp, adb pull /sys/fs/pstore
it will grab the whole folder for you
in the meantime, i have made a flashable tar for kernel with the boot image you gave me. i trimmed out much of the secure stuff & knox, and enabled adb debugging.
https://idlekernel.com/nethunter/herolte/AP_nethunter_G930F_APB9_test1.tar
you should hopefully be able to adb in and get logcat and /proc/kmsg while it is stuck in samsung logo after flashing it....

jcadduono said:
yes it should exist for a few minutes after exiting boot loop and entering twrp, adb pull /sys/fs/pstore
it will grab the whole folder for you
in the meantime, i have made a flashable tar for kernel with the boot image you gave me. i trimmed out much of the secure stuff & knox, and enabled adb debugging.
https://idlekernel.com/nethunter/herolte/AP_nethunter_kernel_G930F.tar
you should hopefully be able to adb in and get logcat and /proc/kmsg while it is stuck in samsung logo after flashing it....
Click to expand...
Click to collapse
Going to do this later this day.

Activadee said:
Going to do this later this day.
Click to expand...
Click to collapse
Ok, lets begin.
I flashed first the test 2 zip from yesterday to get /sys/fs/pstore
flashed the zip via twrp -> reboot to system -> again it dont go over the FIRST Samsung logo - it shows up for 2 secs and rebooting again. This is the only result i get when i flash the zip. after i rebooted to twrp i connected my phone to my laptop to pull /sys/fs/pstore
So i started adb and typed in: adb pull /sys/fs/pstore
the result is
pull: building file list...
0 files pulled. 0 files skipped.
Click to expand...
Click to collapse
There is no files in the folder.
I restored the original boot partition and flashed your *.tar file via odin 3.10.7.
the result is just the same.
For better undestanding i uploaded a picture with my "high end build in 2MP Tablet Camera" to show you what samsung logo i mean. Its showing up for 2 secs, reboots and showing up again.
https://drive.google.com/open?id=0B9IHgLrX7UgTbkstc0RrOS1zamc
With the original boot partition - there is an other samsung logo after the first one. I uploaded it here :
https://drive.google.com/open?id=0B9IHgLrX7UgTcG1nZ3cyZ3VNbkk
Hope i could help you with this.

Activadee said:
Hope i could help you with this.
Click to expand...
Click to collapse
Thanks for your time and assistance. Would anyone else be willing to come to #nethunter on freenode IRC and flash a few test zips for me?

jcadduono said:
Thanks for your time and assistance. Would anyone else be willing to come to #nethunter on freenode IRC and flash a few test zips for me?
Click to expand...
Click to collapse
I would. I have the G935F so could help with that. Let me know. Everytime I go to #nethunter you're not active. I'm assuming it's due to our different time zones. I was then told I shouldn't be there if it's nothing to do with AOSP, which was rude. but anyway, let me know what I can do to help. I am available anytime after 2pm (GMT)
Sent from my SM-G935F using Tapatalk

Just ordered mine unnfortunately there's a 7 day lead time / back log if you're still stuck then I will be happy to come on IRC and help with the testing

this is kind of irrellevant, but could you please trhy to port it to the s6? it has no problems with the bootloader when loading unofficial files. also, i have not planned on getting the s7

Mgrev said:
this is kind of irrellevant, but could you please trhy to port it to the s6? it has no problems with the bootloader when loading unofficial files. also, i have not planned on getting the s7
Click to expand...
Click to collapse
i'm actually having the exact same problem on Note 5 right now, which will also apply to S6...can't figure out how to boot custom kernels without it rebooting at lock screen. =(
hopefully t-mobile S7/edge will not have these problems since it is qcom...
got to say development would be a million times easier if i had these phones myself ><
update 2016-03-28: try this if you're crazy enough! https://idlekernel.com/nethunter/herolte/ (grab latest)

[RECOVERY][pme] TWRP touch recovery

Code:
[CENTER]*** Disclaimer ***
All flashing is done at your own risk!
While nothing from this thread should break your device,
don't come back here blaming anyone if it does![/CENTER]
Introduction
Team Win Recovery Project 3.x, or twrp3 for short, is a custom recovery built with ease of use and customization in mind. Its a fully touch driven user interface no more volume rocker or power buttons to mash. The GUI is also fully XML driven and completely theme-able. You can change just about every aspect of the look and feel.
Click to expand...
Click to collapse
Images
Installation instructions
NOTE: Read the FAQ from Post #2 to ensure that you're installing the correct version of TWRP!!
TWRP Image Install method:
Most devices can be updated quickly and easily within TWRP if you already have version 2.8.4.0 or higher installed.
Download the latest version of TWRP appropriate for your device/firmware
Reboot to TWRP
Hit Install and tap the "Install Image" button in the lower right
Browse to the location of the TWRP image on your device and select it
Select recovery from the partition list and swipe to flash
Alternate Installation Method:
Fastboot Install Method:
You will need the platform-tools from the Android SDK on your computer. Find the Android command line tools section on the page linked and install the SDK tools package. From the SDK Manager, download only the platform-tools to get adb and fastboot binaries.
Windows users will need proper drivers installed on their computer. You can try the Naked ADB drivers or the Universal ADB drivers if you don't already have a working driver installed
On your device, go into Settings -> About and find the Build Number and tap on it 7 times to enable developer settings. Press back and go into Developer Options and enable USB debugging. From your computer, open a command prompt and type:
Code:
adb reboot download
You should now be in fastboot mode.
Download the correct image file and copy the file into the same folder as your adb and fastboot binaries. Rename the image to twrp.img and type:
Code:
fastboot flash recovery twrp.img
Code:
fastboot reboot
Click to expand...
Click to collapse
Device Changelog
Current version: 3.4.0-0:
Code:
[LIST][URL="https://github.com/TeamWin/android_device_htc_pme/commit/f168dc3cd98bb8778e12b14716e3015b9b873256"]Add vendor init[/URL]
[*][URL="https://github.com/TeamWin/android_device_htc_pme/commit/4b4e1c14b65aa3d974e99d08ad0851b5ec24e0c5"]Decryption updates & cleanup[/URL][/LIST]
Older Device-specific versions:
Code:
[SIZE="4"][COLOR="Green"]3.2.3-1:[/COLOR][/SIZE]
[LIST]Updates to support AOSP Pie decryption[/LIST]
[SIZE="4"][COLOR="Green"]3.2.2-1:[/COLOR][/SIZE]
[LIST][update] Add support for AOSP Oreo decryption[/LIST]
[SIZE="4"][COLOR="Green"]3.2.1-4:[/COLOR][/SIZE]
[LIST]Enable f2fs support
- Fixed source so it compiles properly[/LIST]
[SIZE="4"][COLOR="Green"]3.2.1-3:[/COLOR][/SIZE]
[LIST]Update kernel to custom Oreo built from 3.16.708.3_R HTC Dev source
- Patched for proper working touch (reboot recovery now works as well)
[*]Enable NTFS
- f2fs remains disabled, as source won't compile with it enabled[/LIST]
[SIZE="4"][COLOR="Green"]3.2.1-2:[/COLOR][/SIZE]
[LIST]Use /persist as Qualcomm time fix source during early boot
- Fixes broken time issue on Oreo firmware[/LIST]
[SIZE="4"][COLOR="Green"]3.2.1-1:[/COLOR][/SIZE]
[LIST]Updated kernel to US Unlocked Oreo (3.16.617.2) - patched for working touch
[*]Added support for Oreo decryption (posthumous thanks to @nkk71 for all his hard work on decryption)
[*]Disable f2fs & NTFS support until custom kernel can be built
[*]Update vendor init to properly detect Verizon model by CID[/LIST]
Click to expand...
Click to collapse
TWRP Official Changelog
Current version: 3.4.0:
Code:
System As Root (SAR)
[LIST]Fix backup and restore using SAR - dianlujitao
[*]System mount point - Chaosmaster
[*]ORS - Chaosmaster
[*]Zip install - Chaosmaster
[*]system_root bind mount to /system - Chaosmaster
[*]Autodetection of SAR - Chaosmaster[/LIST]
Digest
[LIST]fix creation of digests for sub-partitions (was bugfix applied to many devices since last year) - Bigbiff[/LIST]
Encryption
[LIST]ext4Crypt Wrapped Key Update - Peter Cai
[*]Fix upgrading encryption key if export fails - Peter Cai
[*]Fix wrapped key support for devices without metadata partition - mauronofrio
[*]Don't skip decryption when using block map file in order to write to /data in ORS - CaptainThrowback
[*]FDE - Decrypt master key first - AndroidableDroid
[*]vold_decrypt - set Android version and patch level automatically - CaptainThrowback
[*]Set wrapped decrypt support by twrp flag - Peter Cai
[*]Don't try wrapped support unless needed - mauronofrio
[*]restore ext4 policy on /data/cache - Bigbiff
[*]multiuser decryption - Noah Jacobson
[*]FDE retry - AndroidableDroid[/LIST]
TWRP App
[LIST]unmount system after checking for app - Bigbiff[/LIST]
Prebuilt updates
[LIST][email protected] - cryptomilk[/LIST]
Compilation Fixes
[LIST]TW_EXFAT_FUSE compilation fixes - Bigbiff
[*]libuuid - cryptomilk
[*]'system/etc/ld.config.txt' not found error - Martin Dünkelmann[/LIST]
Language Updates
[LIST]Portugal - Vasco Machado
[*]Dutch - Ian Macdonald
[*]Turkish - Fatih Fırıncı
[*]Localisation of Backup_Tar - Ian Macdonald[/LIST]
ld.config.txt
[LIST]updates for 8.x trees - CaptainThrowback
[*]fix search path for /sbin - CaptainThrowback
[*]/sbin should come first in search path - Ian Macdonald[/LIST]
General Bugs
[LIST]Fix persistent log storage - SyberHexen
[*]Compress Persistent Logs - Bigbiff
[*]FB2PNG compilation errors - Bigbiff
[*]exclude per_boot from backups - Darth9
[*]Unmount all directories that point to same block device - AndroidableDroid
[*]Blank screen fixes - Sean hoyt
[*]Toolbox is default on android-9+ - mauronofrio[/LIST]
Cleanup
[LIST]Typo fix in comment - VDavid003
[*]newlines in ext4crypt - CaptainThrowback
[*]TW_OEM_BUILD compilation issue - Patrick Zacharias
[*]Fix Dependency requirements - Dees_Troy
[*]Fix Symbolic links for BB and Toolbox - Dees_Troy[/LIST]
Bootloader Message
[LIST]cleanup - Alessandro Astone
[*]add configurable offsets[/LIST]
Error Cleanup
[LIST]uevent errors and decryption error - mauronofrio
[*]using copy_file to copy files from /etc - CaptainThrowback
[*]ueventd access to /acct - early directory creation in init - cryptomilk[/LIST]
Haptics
[LIST]TSP Driver - LameMonster82
[*]QTI Input - AndroidableDroid[/LIST]
update_engine
[LIST]read all asserts - Hernán Castañón[/LIST]
Resetprop
[LIST]Add Resetprop from Magisk - CaptainThrowback & mauronofrio
[*]compile from source - Chaosmaster
[*]fix for android-7 and earlier - Chaosmaster
[*]cleanup for spaces in properties - AndroidableDroid[/LIST]
Properties
[LIST]Add Property override - Chaosmaster[/LIST]
Backuptool
[LIST]mount system and vendor for A/B installs for backuptool - Chaosmaster[/LIST]
twrpTar
[LIST]fix backup freezes when pigz and openaes are used - Fabrice Bellet[/LIST]
Zip Installs
[LIST]Info for A/B zip installing to inactive slot - Chaosmaster
[*]Reboot to system button now allows to be rebooted to different partitions after zip install
[*]progressbar rework - Chaosmaster[/LIST]
Magisk updates
[LIST]update binaries from source - AndroidableDroid[/LIST]
A/B Updater Zip Template
[LIST]rewrite A/B installer zip from scratch using a new generic template and latest magiskboot - osm0sis
[*]installer zip support for recovery_a/recovery_b partition ramdisks on newer 2SI SAR A/B devices - osm0sis
[*]generate installer zips for all prod A/B devices - bigbiff
[*]improve installer zip dump/write speed and add more error catching - arter97 & osm0sis[/LIST]
OZIP Encryption Support
[LIST]add OZIP encryption - mauronofrio[/LIST]
File Selector
[LIST]Support for more extensions in File Selector - mauronofrio[/LIST]
Older versions:
Code:
[SIZE="4"][COLOR="Green"]3.3.1:[/COLOR][/SIZE]
[LIST]Fix selinux issues during formatting - dianlujitao
[*]Various fixes for toybox and toolbox builds - CaptainThrowback and bigbiff
[*]Flash both A and B partitions when installing a recovery ramdisk - Dees_Troy
[*]Add option to uninstall TWRP app from /system - Dees_Troy
[*]Create digest for subpartitions - bigbiff[/LIST]
[SIZE="4"][COLOR="Green"]3.3.0:[/COLOR][/SIZE]
[LIST]Merge AOSP 9.0 r3 (Dees_Troy)
[*]Use ANDROID_ROOT variable instead of hard coding to /system (CaptainThrowback)
[*]Decrypt FBE on 9.0 and metadata decrypt (Dees_Troy)
[*]vold decrypt updates (nijel8, CaptainThrowback)
[*]Support vibration on LED class devices (notsyncing)
[*]Metadata decrypt support for Pixel 3 (Dees_Troy)
[*]Support rotating the display via build flag (vladimiroltean)
[*]Reboot to EDL mode button (mauronofrio)
[*]Support MTP on FFS devices (bigbiff)
[*]Update FDE decrypt to support keymaster 3 and 4 (Dees_Troy)
[*]Detect mkfs.f2fs version to properly format on f2fs partitions (Dees_Troy)
[*]Allow TWRP to use md5 and sha256 checksums for zip installs (bigbiff)
[*]TWRP can use /data/cache/recovery and /persist/cache/recovery on AB devices with no cache partition (bigbiff)
[*]Switch part of advanced menus in TWRP to use a listbox of options (Dees_Troy)
[*]Use magiskboot to allow repacking boot images for installing TWRP (Dees_Troy with thanks to topjohnwu of course)[/LIST]
[SIZE="4"][COLOR="Green"]3.2.3:[/COLOR][/SIZE]
[LIST]Fix automatic installing of OTA zips on encrypted devices
[*]Remove SuperSU from TWRP
[*]Support both md5 and md5sum file extensions when doing MD5 checking for zip files[/LIST]
[SIZE="4"][COLOR="Green"]3.2.2:[/COLOR][/SIZE]
[LIST]adb backup fixes
[*]OTA style update zips will now install automatically without prompting for decrypt
[*]minor tweaks to handling date/time on Qualcomm devices
[*]updates to some language translations[/LIST]
[SIZE="4"][COLOR="Green"]3.2.1:[/COLOR][/SIZE]
[LIST]minui fixes (cryptomilk)
[*]Better android-8.0 compatibility in ROM trees (Dees_Troy)
[*]Fix missing library in android-8.0 (nkk71)
[*]Fix inconsistent SDCard naming (DevUt)
[*]Default to TWRP restore instead of adb backup restore to fix restore on fresh TWRP boot (jlask)[/LIST]
[SIZE="4"][COLOR="Green"]3.2.0:[/COLOR][/SIZE]
[LIST]Allow restoring adb backups in the TWRP GUI (bigbiff)
[*]Fix gzip backup error in adb backups (bigbiff)
[*]Fix a bug in TWRP's backup routines that occasionally corrupted backup files (nkk71)
[*]Better support for installing Android 8.0 based zips due to legacy props (nkk71)
[*]Support vold decrypt with keymaster 3.0 in 8.0 firmwares (nkk71)
[*]Decrypt of synthetic passwords for Pixel 2 (Dees_Troy)
[*]Support newer ext4 FBE policies for backup and restore in libtar (Dees_Troy)
[*]v2 fstab support (Dees_Troy)
[*]Bring TWRP forward to android 8.0 AOSP base (Dees_Troy)
[*]Various other minor bugfixes and tweaks[/LIST]
[SIZE="4"][COLOR="Green"]3.1.1:[/COLOR][/SIZE]
[LIST]Backups will now include adopted storage keys (Dees_Troy)
[*]Fixed an adb restore issue (bigbiff)
[*]Fixed rebooting when no OS is present (Dees_Troy)
[*]Fixed line wrapping in the GUI terminal (_that)
[*]Updated TWRP source code to AOSP 7.1.2 (Dees_Troy)[/LIST]
[SIZE="4"][COLOR="Green"]3.1.0:[/COLOR][/SIZE]
[LIST]vold decrypt on a few select HTC devices, TWRP will now attempt to use the system partition's vold and vdc binaries and libraries to decrypt the data partition (nkk71 and CaptainThrowback)
[*]adb backup to stream a backup directly to or from your PC, see documentation [URL="https://github.com/omnirom/android_bootable_recovery/commit/ce8f83c48d200106ff61ad530c863b15c16949d9"]here[/URL] (bigbiff)
[*]tweak MTP startup routines (mdmower)
[*]support new Android 7.x xattrs for backup and restore to fix loss of data after a restore (Dees_Troy)
[*]support POSIX file capabilities backup and restore to fix VoLTE on HTC devices and possibly other issues (Dees_Troy)
[*]better indicate to users that internal storage is not backed up (Dees_Troy)
[*]improve automatic determination of TW_THEME (mdmower)
[*]minimal getcap and setcap support (_that)
[*]try mounting both ext4 and f2fs during decrypt (jcadduono and Dees_Troy)
[*]shut off backlight with power key (mdmower)
[*]timeout during FDE decrypt (Dees_Troy and nkk71)
[*]support for FBE decrypt and backing up and restoring FBE policies (Dees_Troy)
[*]boot slot support (Dees_Troy)
[*]TWRP app install prompt during reboot (Dees_Troy)
[*]support for AB OTA zips (Dees_Troy)
[*]support new Android 7.x log command (Dees_Troy)
[*]update recovery sources to AOSP 7.1 (Dees_Troy)
[*]numerous bugfixes and improvements by too many people to mention[/LIST]
[SIZE="4"][COLOR="Green"]3.0.3:[/COLOR][/SIZE]
[LIST]Partial release to help support the release of the [URL="https://www.xda-developers.com/team-win-releases-their-first-official-twrp-app-in-the-play-store/"]Official TWRP app[/URL][/LIST]
[SIZE="4"][COLOR="Green"]3.0.2:[/COLOR][/SIZE]
[LIST]Fix a bug with the input box that affected masked inputs (passwords). This fixes decrypt of full device encryption on devices that support decrypt. This bug also impacts encrypted backups. Users are highly encouraged to stop using 3.0.1 if you use encrypted backups or if you need decrypt of data in TWRP.
[*]Add Greek translation to some builds.[/LIST]
[SIZE="4"][COLOR="Green"]3.0.1:[/COLOR][/SIZE]
[LIST]support new CM 13.0 pattern encryption (sultanqasim)
[*]fix slow flashing issue due to modprobe (present on only some devices) (#twrp)
[*]libtar updated to latest upstream and fixes (jcadduono)
[*]fixes for loading custom themes (_that)
[*]TWRP will now detect and install TWRP themes automatically through the normal zip install process (Dees_Troy)
[*]translation updates - added Italian, Czech and Polish and significant updates to Dutch
[*]progress bar improvements - progress bar updates during image flashing and better tracks progress during file system backups (tar) (Dees_Troy)
[*]fix input box text display (Dees_Troy)
[*]reboot option after zip install complete (bigbiff)
[*]other mostly invisible bug fixes and improvements[/LIST]
[SIZE="4"][COLOR="Green"]3.0.0:[/COLOR][/SIZE]
[LIST]Completely new theme - Much more modern and much nicer looking (by z31s1g)
[*]True Terminal Emulator - Includes arrow keys, tab and tab completion, etc. (by _that)
[*]Language translation - It won’t be perfect and especially some languages that require large font files like Chinese & Japanese won’t be availble on most devices. Also some languages may only be partially translated at this time. Feel free to submit more translations to OmniROM’s Gerrit. (mostly by Dees_Troy)
[*]Flashing of sparse images - On select devices you will be able to flash some parts of factory images via the TWRP GUI (by HashBang173)
[*]Adopted storage support for select devices - TWRP can now decrypt adopted storage partitions from Marshmallow
[*]Reworked graphics to bring us more up to date with AOSP - includes support for adf and drm graphics (by Dees_Troy)
[*]SuperSU prompt will no longer display if a Marshmallow ROM is installed
[*]Update exfat, exfat fuse, dosfstools (by mdmower)
[*]Update AOSP base to 6.0
[*]A huge laundry list of other minor fixes and tweaks[/LIST]
[U]Additional Notes[/U]
[LIST]WARNING: This is our first release in a long time. We have a lot of new and somewhat aggressive changes in this new release. The changes to the graphics back-end may cause some devices to not boot up properly or have other display-related issues. If you are not in a position to reflash an older build of TWRP, then wait until you are or at least wait until others have tried the new version for your specific device. You don’t want to end up with a non-working recovery and have to wait several hours or days to get to a computer to be able to fix it.
[*]Notes for themers: In addition to the updated theme, we have introduced a theme version variable to the TWRP theme system. If the theme version does not match the version that TWRP expects, TWRP will reject the custom theme and load its stock theme. This change will ensure that people who update TWRP without updating their theme will still have a workable recovery. We have removed libjpeg support. The stock theme was only using a jpeg image for the splash / curtain. This change means that any custom themes will no longer be able to use jpeg images. It also means that tools used to repack recovery images with a different curtain / splash will need to be updated to use the new method.
[*]Version number notes: For a while we’ve been using a 4 digit version number and reserved the 4th digit for device-specific updates. For instance, we find and fix a device-specific issue like decryption of data on Nexus 5, we would release that as a 2.8.7.1. After a while, some people would start asking where 2.8.7.1 was for other devices. So, going forward we have decided to change the numbering scheme to 3.0.0-2, etc. Our hope is that this version numbering scheme will more clearly identify that the 4th digit does not indicate a version change for the code base.
[*]We need your help! The bulk of TWRP work is done by 3 people on a volunteer basis. We have pushed most of our device files to our github and we have a gerrit instance. If you have the ability, please help us maintain our official devices and/or add your device to our official device list. Thanks in advance![/LIST]
[SIZE="4"][COLOR="Green"]2.8.7.0:[/COLOR][/SIZE]
[LIST]Initial ground work for software drawn keyboard (_that)
[*]Fix handling of wiping internal storage on datamedia devices (xuefer)
[*]Allow DataManager to set and read values from the system properties (xuefer)
[*]Fix crash when taking screenshots on arm64 devices (xuefer)
[*]Fix error message after an ORS script completes (Dees_Troy)
[*]Fix crashes / error when creating encrypted backups (_that, Dees_Troy)
[*]Add system read only option – more details below (Dees_Troy)
[*]Add resize2fs and GUI option to run resize2fs (Dees_Troy)
[*]Fix crash loop caused by empty lines in AOSP recovery command file (_that)
[*]Prevent duplicate page overlays such as multiple lock screens (mdmower)[/LIST]
[U]Additional Notes[/U]
[LIST]Note: As always, be sure your custom theme is up to date (or remove your custom theme) before updating TWRP.
[*]System read only option: Devices that ship with 5.0 and higher as their initial OS are using block level OTA updates. With this style of OTA update, the update script checks to see if the system partition has ever been mounted read/write. Further, the script also usually runs an SHA sum of the entire system partition to detect if any changes have been made. If any changes have been made, the OTA update will refuse to install. Since not all OEMs and devices have factory images available, we have created a new feature in TWRP that detects if the system partition has ever been mounted read/write. If not, you will be prompted asking if you want TWRP to mount system as read/write. If you choose not to allow TWRP to mount as read/write, TWRP won’t prompt to install SuperSU and TWRP won’t try to patch the stock ROM to prevent TWRP from being replaced by stock recovery. The goal of this option is to hopefully allow the user to make a raw system image backup that they can use to get back to a state where they can take OTA updates again.
[*]resize2fs feature: On some devices like the Nexus 6, the factory images include a userdata image that is the proper size only for the 32GB units. If you flash the factory image to a 64GB Nexus 6, the data partition will appear as if it only has the free space of a 32GB device. Using the resize2fs option, TWRP can resize your data partition to take up the full space available. The resize2fs may also be useful to resize system partitions on devices where custom ROM system images don’t take up the full partition space. Lastly, resize2fs may be useful in some cases to reserve the proper space at the end of a data partition for a full disk encryption key, should your partition be formatted incorrectly for some reason.
[*]This new version also marks our first set of full builds using our new jenkins build server. You can track the progress of builds at [url]https://jenkins.twrp.me[/url] and we have taken additional steps to make it easier for device maintainers to step up and submit patches to our gerrit server at [url]https://gerrit.twrp.me[/url] to help us keep devices up to date and working.[/LIST]
Click to expand...
Click to collapse
Downloads
NOTE: Read the FAQ from Post #2 to ensure that you're installing the correct version of TWRP!!
Download
Latest Official versions
Latest Unofficial versions
Sources
Device tree
Kernel source
Click to expand...
Click to collapse
FAQ - Post #2
Known Issues
As of version 3.3.0, stock Nougat can no longer be decrypted. Use 3.2.3 or older if you are still running stock Nougat.
Encrypted backups are broken - DO NOT USE THIS FEATURE!!
3.2.1-1 through 3.2.1-2: Reboot recovery is broken (due to patching stock kernel for touch - requires kernel source and custom kernel build to fix) - UPDATE: Fixed with 3.2.1-3
Click to expand...
Click to collapse
We need your help!
Join the TWRP Testing group on Slack to help us test TWRP prior to official releases!
Click to expand...
Click to collapse
Bug Reporting
If you have an issue, the first step is to post a recovery log so we can determine the cause of the issue. This is done in recovery using Advanced -> Copy Log, or adb pull /tmp/recovery.log. Once a log is uploaded we can determine how best to proceed. NOTE: Posts that are reporting bugs or issues without an accompanying recovery log will be ignored! Additionally, providing details about your device setup, including variant, firmware version, and exact steps to reproduce your issue will also be helpful in diagnosing the problem.
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
If your issue is determined to be a bug by the device maintainer, please consider posting it to our github issues log. It's pretty much impossible for us to keep up with the more than 40 threads that we have for the devices that we "directly" support. If you have a significant problem that cannot be answered in this thread, your best bet is to contact us via our website, or find us in our IRC channel below. If you see someone that's struggling, feel free to point it out to us. We need your help to help us keep track of all of our devices! Thanks!
Click to expand...
Click to collapse
Additional Help/Support:
Live support is available via #twrp on Freenode with your IRC client or just click this link.
Click to expand...
Click to collapse
XDA:DevDB Information
TeamWin Recovery Project (TWRP), Tool/Utility for the HTC 10
Contributors
Captain_Throwback, Dees_Troy, bigbiff, _that, nkk71
Source Code: https://github.com/TeamWin/android_bootable_recovery
Version Information
Status: Stable
Current Stable Version: 3.4.0-0
Stable Release Date: 2020-06-24
Created 2016-04-13
Last Updated 2020-06-25

Frequently Asked Questions (FAQ)
1. Why is this device different than my previous HTC device?
With the 10 (and the M9 and A9 prior to it), HTC has moved to a block-based OTA system. This means that even mounting system as read-write (as TWRP typically does during startup checks) will nullify the device's ability to take an OTA. Any other changes to the system partition will also cause an OTA to fail (even if that check is removed from the OTA zip) due to "unexpected contents." Additionally, see this thread.
2. I decrypted my device and now I don't have signal. Did a TWRP update cause this?
No, this is not a TWRP issue. It appears that on the HTC 10, having a decrypted device prevents the SIM card/telephony from initializing, resulting in no mobile signal or data connection. The device comes with forced encryption enabled and currently must remain that way in order for mobile signal/data to work. TWRP does not change or affect any of that.
3. Which version of TWRP am I supposed to use?
For all ROMs except stock-based Nougat ROMs: The best version to use is 3.4.0-0, the latest official TWRP from twrp.me.
For stock-based Nougat ROMs: The best version to use is 3.2.3-0, from twrp.me.
4. Why is there a "System" backup option and a "System Image" backup option now?
The "System" option is the standard tar backup. "System Image" is a dd backup of the entire system block device (/dev/block/bootdevice/by-name/system). The "System Image" option is only relevant if your system is unmodified. This allows you to make a fully stock backup that can be restored later to take an OTA.
NOTE: You only need to choose ONE of these options when making a backup!!
[*]NOTE 2: If you are using a FAT32-formatted card, a "System Image" backup may fail (depending on your variant), due to the 4GB file limit on that format. For a successful System Image backup, internal storage or NTFS/exFAT-formatted external storage must be used (either SD card or USB-OTG)
5. How am I supposed to root?
Since the 10 has dm-verity enabled and forces encryption by default, root can only be achieved using a "systemless" root method. Magisk is the recommended root solution, as it is actively developed and up-to-date. It also allows devices to pass Google's SafetyNet API for working contactless payments. See the below thread for full details.
Magisk
6. How do I backup stock recovery prior to flashing TWRP?
You can't. The "fastboot boot" command appears to be disabled on the 10's ABOOT, so TWRP must be fastboot flashed over stock recovery. You can however, extract the stock recovery.img from the OTA firmware.zip when it's received and use that to install the OTA.
An alternate method to obtain a stock recovery is listed below, but it requires 2 devices (either owned by you, or help from someone else in the forum):
Someone fastboot flash twrp and immediately make a backup of boot and upload it to XDA.
Once the above is available, someone else download that boot.img to their device, and fastboot flash twrp to the BOOT partition of their device.
Once the above is done, reboot the device, which will bring up TWRP, and then backup stock RECOVERY in TWRP, and upload to XDA.
Then, from within TWRP, use the Image install feature in TWRP to flash the stock boot.img.
7. How do I restore stock system so that I can accept an OTA?
Check the "Mount system as read-only" box in the Mount menu.
Restore stock "System Image" backup (This will only work if you've made a System Image backup prior to making any modifications to /system).
Fastboot flash stock recovery (fastboot flash recovery recovery_signed.img)
NOTE: It is also possible to restore stock recovery via the TWRP GUI. Rename the stock recovery file to "recovery.emmc.win" and place in the backup folder with the stock system image. Recovery will then show as a restore option. MAKE SURE YOU REALLY WANT TO DO THIS, AS TWRP WILL BE GONE WHEN YOU REBOOT OUT OF RECOVERY!!
[*]NOTE 2: It is possible to install an OTA without using stock recovery (i.e. installing it with TWRP). TWRP will not flash the firmware.zip included in an OTA file. Please see here for a detailed description of the process.
Reboot to system, install OTA.
8. What if I have an RUU? Do I need to worry about all this OTA nonsense?
Not if you don't care about losing all your data. If you're S-ON and have an RUU available for your exact variant (model ID and CID must match) and software number (main version must be the same or newer), then you can get back to a fully stock state by relocking (fastboot oem lock) and flashing an RUU. However, if you'd prefer to take an OTA to keep your data intact, the method stated above is how to do so. Or, you can just run a custom ROM and wait for your ROM chef to update their ROM to the latest software (though you'll still have to find a way to update your firmware if you're not S-OFF)
9. After I go through all this and successfully apply an OTA, how do I make sure I have a clean starting point again?
After the OTA is applied and TWRP is flashed, it will once again detect an untouched system, which will mount system read-only and allow you to make a fully stock backup and start the process over again, this time with the new base.
10. After I restored my Data backup and boot back to Android, I'm entering the correct PIN/password, but it's telling me the password is wrong. What happened, and how do I fix it?
It appears that sometimes after restoring a backup of Data where security was enabled (such as a PIN or password lock), the device does not recognize the correct password. There are two ways to avoid this issue:
Disable security in Android before making a backup of data.
After restoring Data, while still in TWRP, use the TWRP File Manager to navigate to /data/system and delete all the locksettings.* files (such as locksettings.db, etc). When you reboot, the password will be gone.
11. Information about encrypted devices (by default all HTC 10 devices on Nougat+ are encrypted using FDE force-encrypt)
TWRP decryption on the HTC 10 on Nougat+ relies on the currently running ROM's own system files, to be able to decrypt your data partition.
If you intentionally or accidentally delete your system partition, and boot into TWRP without the needed system files, decryption will fail and you will be prompted to enter your password, which will continually fail due to the missing system files.
When the TWRP console is shown during decryption, you will see a red text: "Missing files needed by vold decrypt: /system/bin/vold".
If you encounter this situation, do not panic, do not format your data partition, you will most likely be able to decrypt again, once you have the needed setup back in place, without any data loss.
The easiest way to do this:
Cancel the decrypt prompt
Restore your last System or System_Image backup (no other partitions need to be restored!)
Reboot to bootloader and then back to TWRP (since currently the direct reboot to recovery is broken due to the lack of kernel source code)
Alternatively, in case you do not have a backup available, you can dirty flash whatever ROM you are running, or even fastboot flash a proper system.img, but considering that your system partition is not likely to have changed, since these days most things are run systemless, the restore is the easiest and fastest.

PSA for Devs regarding mounting system in updater-scripts
In order for ROM zips to function properly, ROM devs may need to change the way they're used to performing mount commands in their updater-scripts. Many ROM devs like to use busybox to mount system, like this:
Code:
run_program("/sbin/busybox", "mount", "/system");
However, on newer HTC devices, this will not always be effective, due to the variable state of system when booting into TWRP (more information in the above post). The most reliable way to mount system in TWRP is to use the below command:
Code:
run_program("/sbin/mount", "-t", "ext4", "/dev/block/bootdevice/by-name/system", "/system");
Since this calls the mount binary and specifies the proper filesystem of /system, it will work regardless of the mount state of /system within TWRP.
The reason people are having issues booting ROMs flashed without the above command is because custom ROMs typically include a command to format system within the updater-script, like this one:
Code:
run_program("/sbin/mke2fs", "-t", "ext4", "-m", "0", "/dev/block/bootdevice/by-name/system");
Notice that this calls the mke2fs binary directly and specifies the filesystem, so this command will run properly and format system. However, when the script attempts to mount system using the busybox mount command, it's unsuccessful, resulting in a failed ROM install (even though there likely isn't an error message) and an empty system partition. The same command format should be used both to format system and to mount it.
So, the solution is easy - replace your busybox mount commands with the direct version noted above. Running it that way should work regardless of the update-binary used, since it's not running mount directly but calling it (this makes a difference to some binaries for some reason). This will even work in situations where system has been mounted read-only by TWRP (more on that in the below post).
ROM Devs - please consider making this change going forward. The above method will ALWAYS WORK. Please use this and discontinue whatever other methods to mount system that you have previously used, as they will have inconsistent results, leading people to flash old versions of TWRP or blame TWRP for their errors.
I know HTC is changing how they do things, and it's different than we're all used to. Let's all adapt to the change together. .

You may notice that there's no device page for the HTC 10 on the twrp.me site. Well, obviously, that's because the device hasn't been released yet. However, if you happen to have the device already, you may be able to find a TWRP build for it somewhere, if you figure out where to look .
Once the device is released, please be assured that a device page will be created so everyone can easily get to it. Until that happens, I'll be keeping the thread closed. Thanks for your understanding!

Alright, since @topjohnwu was kind enough to post a TWRP backup from his release version device, I'm opening the thread and offering an "unofficial" release on the Downloads tab until I can get the official build working.
Here's a link to the build:
twrp-3.0.2-1-pme - DEPRECATED BY OFFICIAL RELEASE
P.S. If you happen to have the currently elusive Sprint variant of the 10, this version WILL NOT WORK. I do have a working version for Sprint, but I will need to wait for the release software before I post it.
UPDATE: As of the official 3.0.2-2 release, the Sprint variant is supported!

Captain_Throwback said:
Alright, since @topjohnwu was kind enough to post a TWRP backup from his release version device, I'm opening the thread and offering an "unofficial" release on the Downloads tab until I can get the official build working.
Here's a link to the build:
twrp-3.0.2-1-pme
P.S. If you happen to have the currently elusive Sprint variant of the 10, this version WILL NOT WORK. I do have a working version for Sprint, but I will need to wait for the release software before I post it.
Click to expand...
Click to collapse
Thanks a lot man. I've got a minor issue here, take a look at the picture, it has a mouse cursor in the center for all time.
Also, would you mind me contact you on hangouts? I am curious about some questions, and I cannot contact using PM.

topjohnwu said:
Thanks a lot man. I've got a minor issue here, take a look at the picture, it has a mouse cursor in the center for all time.
Also, would you mind me contact you on hangouts? I am curious about some questions, and I cannot contact using PM.
Click to expand...
Click to collapse
You're just going to have to ignore the mouse cursor for now. I can't get get rid of that without kernel source.
Sure you can contact me on Hangouts. I do prefer PM, though.
P.S. Added mouse cursor issue to current bug list.

I'm trying to do some preliminary setup for AOSP encryption support. Can someone with a device please try flashing the 3.0.2-2 version I just posted, and provide the recovery.log afterwards (provided it boots)? I'd really appreciate it. Thanks!

Better yet, if someone can flash this 3.0.2-3 and pull a recovery log, and also post the output of lsmod either from adb shell or the TWRP Terminal, I'd appreciate it. I'm trying to a few things for decryption.

@Captain_Throwback
Tried 3.0.2-2 and 3.0.2-3, both cannot boot. Is there a way to pull logs?

Captain_Throwback said:
Better yet, if someone can flash this 3.0.2-3 and pull a recovery log, and also post the output of lsmod either from adb shell or the TWRP Terminal, I'd appreciate it. I'm trying to a few things for decryption.
Click to expand...
Click to collapse
Thanks Captain_Throwback .
twrp-3.0.2-2-pme.img & twrp-3.0.2-3-pme.img is work for me . devices TW 1.21.709.2 deodexed

topjohnwu said:
@Captain_Throwback
Tried 3.0.2-2 and 3.0.2-3, both cannot boot. Is there a way to pull logs?
Click to expand...
Click to collapse
When you say they can't boot, what do you mean? What happens, exactly?
When it tries to boot, does it get stuck? If so, do you have adb access? The OP says how to adb pull a recovery log. If not, then it'll just have to wait until I have one for me to try to fix it.

nenebear said:
Thanks Captain_Throwback .
twrp-3.0.2-2-pme.img & twrp-3.0.2-3-pme.img is work for me . devices TW 1.21.709.2 deodexed
Click to expand...
Click to collapse
Thanks, but it looks like you've already removed encryption, so these logs don't really help for what I was trying to work on. I do appreciate your willingness to help, though.

Captain_Throwback said:
When you say they can't boot, what do you mean? What happens, exactly?
When it tries to boot, does it get stuck? If so, do you have adb access? The OP says how to adb pull a recovery log. If not, then it'll just have to wait until I have one for me to try to fix it.
Click to expand...
Click to collapse
It just blinks the splash screen for about 0.5 sec and then no output. About a few seconds later it reboot to system.
I tried to catch adb shell in that few seconds, but it seems it is too short for TWRP to initialize the adbd

topjohnwu said:
It just blinks the splash screen for about 0.5 sec and then no output. About a few seconds later it reboot to system.
I tried to catch adb shell in that few seconds, but it seems it is too short for TWRP to initialize the adbd
Click to expand...
Click to collapse
Understood. I've seen that behavior before with decryption. That'll be difficult to debug without having to device in-hand, so it'll have to wait. Thanks for your feedback, and for being willing to test!
I'll pull down those other versions for now, since they add nothing but freezes.

I've removed the temporary version from the Downloads tab, as the official build has been updated.
And now we have a device page!
https://twrp.me/devices/htc10.html

edited

Is it possible to enable encryption once TWRP installed and dm-verity is disabled?
I'm talking about the old method of encryption where the /data partition is manually encrypted, rather than forceencrypt
Rooting and modding is great, but if the only way to do so is to disable encryption completely, I think that will change a lot of people's value prop.

orrorin said:
Is it possible to enable encryption once TWRP installed and dm-verity is disabled?
I'm talking about the old method of encryption where the /data partition is manually encrypted, rather than forceencrypt
Rooting and modding is great, but if the only way to do so is to disable encryption completely, I think that will change a lot of people's value prop.
Click to expand...
Click to collapse
TWRP doesn't support HTC's encryption. Not sure why you'd think there's a difference between the forced encryption and manual. They still use the same type of encryption on the stock ROM.
Once AOSP is available, decryption should be possible, but only on AOSP-based ROMs.

Captain_Throwback said:
TWRP doesn't support HTC's encryption. Not sure why you'd think there's a difference between the forced encryption and manual. They still use the same type of encryption on the stock ROM.
Once AOSP is available, decryption should be possible, but only on AOSP-based ROMs.
Click to expand...
Click to collapse
I understand that TWRP can't decrypt HTC's encryption, but what I'm wondering is whether it's possible to enable encryption on the HTC 10 after dm-verity has been disabled and root has been flashed.

Magisk works!! [+ POC boot.img for 3/19/18 LOS 14.1]

Please also read the additional notes in post #2, as they are critical to getting Magisk working.
I decided to do some tinkering around with Magisk, and it actually DOES work on the kindles (at least the 8.9"). The problem is, Magisk's patcher just isolates the ramdisk part of the boot.img and doesn't add the boot signature or other magic back to the image when it's time to reflash the patched boot image. By dd'ing the signature (and other files) back to the image, I can get Magisk to successfully boot.
As part of the working POC (because it's exciting to actually see this!), I've uploaded the patched "Magiskified" boot image (which originally comes from the 20180319 LineageOS 14.1 ROM that was built about a week ago). For reference, this is patched by Magisk v16.0, and the setup is basically the same as the official boot.img makefile directions from CM12.1. (It was the most arbitrary source I found, and I doubt the magic used to create the boot images has changed, so I'm just using that script as a reference.) Try to stick to that ROM if you can - no telling what different ROM versions/variants might do if you're not careful.
I plan on releasing a flashable .zip soon (probably in a month? I have college to work through) to automate the patching process, and possibly even extract the official installer zips to work through Magisk's patching scripts manually so the required boot magic can be patched back into the image before it's ever flashed. (I'll try to take requests to manually patch other ROM boot.imgs if asked to in the meantime though.)
As a friendly reminder, please do NOT flash the official Magisk installer zips or any patched boot images that the app produces as is - they need to be "repatched" with the boot magic, or you'll have to fastboot flash your ROM's boot.img manually because the kindle will hang at the bootloader screen.

Important notes
The official Magisk v16.0 zip must be flashed on first install/reinstall in order to properly construct the environment. Flash the boot image attached in the OP immediately after without rebooting in between, or the image Magisk flashed will prevent the kindle from booting normally without advanced intervention.
SafetyNet does NOT pass the basic integrity OR advanced checks. At least, v16 doesn't. Maybe an earlier Magisk build does - feel free to try it once I get the automated patcher zip up and running.
For now, because you're flashing on LineageOS, you may want to flash the LOS 14.1 arm-based su removal zip from Lineage's downloads site. Verify you're downloading arm and not arm64.

How does one go about patching the boot image thats modified by magisk so it's able to be flashed?

kn0wbodh1 said:
How does one go about patching the boot image thats modified by magisk so it's able to be flashed?
Click to expand...
Click to collapse
It's complicated. I recommend not doing this unless you're willing to follow it to the letter - when I get to creating the automated patcher, this won't be necessary.
Make backups!!
extract the boot.img from your ROM .zip, copy it to the device internal storage
install the Magisk Manager app, download the Magisk .zip and choose "patch boot image"; navigate to said boot image file
copy the modified image to a computer (preferably one running a Linux OS like Ubuntu)
download the boot_cert and u-boot.bin files from the official LineageOS/CM device repo; place these files in the same directory as the boot.img file
open a Linux terminal pointed to the same directory as the boot.img file
run for i in $(seq 1024); do echo -ne "\x00\x50\x7c\x80" >> stack.tmp; done to create the remaining file
run cat boot_cert patched_boot.img > boot.img (assuming the Magisk image produced is named patched_boot.img); this is the boot "signature"
run dd if=u-boot.img of=boot.img bs=8117072 seek=1 conv=notrunc to tag the second bootloader on
finally, run dd if=stack.tmp of=boot.img bs=6519488 seek=1 conv=notrunc to add the stack file; copy the new boot.img back to the kindle
reboot into recovery, flash the Magisk .zip to build the environment, but do NOT reboot yet
choose "Flash .img" within TWRP, select the boot.img, and select "Boot" to flash to the boot partition; reboot to system once complete
profit!

monster1612 said:
It's complicated. I recommend not doing this unless you're willing to follow it to the letter - when I get to creating the automated patcher, this won't be necessary.
Make backups!!
extract the boot.img from your ROM .zip, copy it to the device internal storage
install the Magisk Manager app, download the Magisk .zip and choose "patch boot image"; navigate to said boot image file
copy the modified image to a computer (preferably one running a Linux OS like Ubuntu)
download the boot_cert and u-boot.bin files from the official LineageOS/CM device repo; place these files in the same directory as the boot.img file
open a Linux terminal pointed to the same directory as the boot.img file
run for i in $(seq 1024); do echo -ne "\x00\x50\x7c\x80" >> stack.tmp; done to create the remaining file
run cat boot_cert patched_boot.img > boot.img (assuming the Magisk image produced is named patched_boot.img); this is the boot "signature"
run dd if=u-boot.img of=boot.img bs=8117072 seek=1 conv=notrunc to tag the second bootloader on
finally, run dd if=stack.tmp of=boot.img bs=6519488 seek=1 conv=notrunc to add the stack file; copy the new boot.img back to the kindle
reboot into recovery, flash the Magisk .zip to build the environment, but do NOT reboot yet
choose "Flash .img" within TWRP, select the boot.img, and select "Boot" to flash to the boot partition; reboot to system once complete
profit!
Click to expand...
Click to collapse
Thank you very much for the detailed instructions. I'll be keeping an eye out for the automated patcher you mentioned. Would love to try out magisk on my 2015 fire.

kn0wbodh1 said:
Thank you very much for the detailed instructions. I'll be keeping an eye out for the automated patcher you mentioned. Would love to try out magisk on my 2015 fire.
Click to expand...
Click to collapse
The instructions only work against the 2012 fire (HD 8.9", 2nd generation). They will more than likely brick any other device. I don't recommend trying the instructions unless you're 100% sure your device is that specific model.

Hi, a month ago i flashed oifficial magisk 16 zip on a 8.9 kindle fire hd, and as you said, dont boot anymore, just satys on the kindle fire logo, please can you tell me how can i restore my device?, i havent used it in almost 3 years and i dont have a clue on what to do, i just wanted to install viper4android and now is dead.

erick_gc said:
Hi, a month ago i flashed oifficial magisk 16 zip on a 8.9 kindle fire hd, and as you said, dont boot anymore, just satys on the kindle fire logo, please can you tell me how can i restore my device?, i havent used it in almost 3 years and i dont have a clue on what to do, i just wanted to install viper4android and now is dead.
Click to expand...
Click to collapse
https://forum.xda-developers.com/showthread.php?t=2128848&p=75525760
I know it's not for the 8.9" but I was able to get my 7" working by repeating the procedure in step 5. Magisk messes up the kernel on the Kindle so all you have to do is reflash the kernel. You'll need a fastboot cable to get in fastboot mode though.
Take a look at the few posts before the one I linked to.

just wondering if you've had any luck with the flashable zip for magisk? Not confident enough to try it manually. Thanks in advance.
monster1612 said:
Please also read the additional notes in post #2, as they are critical to getting Magisk working.
I decided to do some tinkering around with Magisk, and it actually DOES work on the kindles (at least the 8.9"). The problem is, Magisk's patcher just isolates the ramdisk part of the boot.img and doesn't add the boot signature or other magic back to the image when it's time to reflash the patched boot image. By dd'ing the signature (and other files) back to the image, I can get Magisk to successfully boot.
As part of the working POC (because it's exciting to actually see this!), I've uploaded the patched "Magiskified" boot image (which originally comes from the 20180319 LineageOS 14.1 ROM that was built about a week ago). For reference, this is patched by Magisk v16.0, and the setup is basically the same as the official boot.img makefile directions from CM12.1. (It was the most arbitrary source I found, and I doubt the magic used to create the boot images has changed, so I'm just using that script as a reference.) Try to stick to that ROM if you can - no telling what different ROM versions/variants might do if you're not careful.
I plan on releasing a flashable .zip soon (probably in a month? I have college to work through) to automate the patching process, and possibly even extract the official installer zips to work through Magisk's patching scripts manually so the required boot magic can be patched back into the image before it's ever flashed. (I'll try to take requests to manually patch other ROM boot.imgs if asked to in the meantime though.)
As a friendly reminder, please do NOT flash the official Magisk installer zips or any patched boot images that the app produces as is - they need to be "repatched" with the boot magic, or you'll have to fastboot flash your ROM's boot.img manually because the kindle will hang at the bootloader screen.
Click to expand...
Click to collapse

barcia99 said:
just wondering if you've had any luck with the flashable zip for magisk? Not confident enough to try it manually. Thanks in advance.
Click to expand...
Click to collapse
You can't directly flash the official installer zips onto the Kindle - they currently bork the boot image "signature" (causing the bootloader exploit to break) and require reflashing the boot image from your ROM via fastboot to get things working again.
What I've thought of is adding some device detection logic to the installer script and then having it run through the process of properly repatching the boot image after the main Magisk install finishes in order to get things to work (as opposed to having a supplementary zip file work through that after an official build is flashed).
I forked the official Magisk repo a while ago and honestly forgot about it, but since v17 hit stable since then, I'm going to rebase those proposed changes against that version. No ETA on that as of yet - I've started back at college, so time is already kind of a rarity; in addition, given the age of the Kindles already (5+ years!), it may not be a thing to sustain long term. I still have my 8.9", so testing isn't an issue, but I don't expect Magisk running on these specific devices to function as expected (so more than likely SafetyNet will fall, probably Magisk Hide as well). I'm not 100% sure how it'll turn out, but these are pretty much going to be unofficial builds for as long as I/anyone else willing to run builds sees a benefit to doing so. When a build works to my satisfaction, I promise it'll go up on XDA.

monster1612 said:
You can't directly flash the official installer zips onto the Kindle - they currently bork the boot image "signature" (causing the bootloader exploit to break) and require reflashing the boot image from your ROM via fastboot to get things working again.
What I've thought of is adding some device detection logic to the installer script and then having it run through the process of properly repatching the boot image after the main Magisk install finishes in order to get things to work (as opposed to having a supplementary zip file work through that after an official build is flashed).
I forked the official Magisk repo a while ago and honestly forgot about it, but since v17 hit stable since then, I'm going to rebase those proposed changes against that version. No ETA on that as of yet - I've started back at college, so time is already kind of a rarity; in addition, given the age of the Kindles already (5+ years!), it may not be a thing to sustain long term. I still have my 8.9", so testing isn't an issue, but I don't expect Magisk running on these specific devices to function as expected (so more than likely SafetyNet will fall, probably Magisk Hide as well). I'm not 100% sure how it'll turn out, but these are pretty much going to be unofficial builds for as long as I/anyone else willing to run builds sees a benefit to doing so. When a build works to my satisfaction, I promise it'll go up on XDA.
Click to expand...
Click to collapse
thank's much. i'll continue to do some research also. i've had this kindle since it came out and remains stable with root and twrp. runs smooth and just plain like it. only negative is no sd card slot. again thanks for your hard work.

Hoping for the automated package
Here's hoping you get time to finish the automated flash package. I am not confident enough to attempt this even with your detailed instructions.
monster1612 said:
You can't directly flash the official installer zips onto the Kindle - they currently bork the boot image "signature" (causing the bootloader exploit to break) and require reflashing the boot image from your ROM via fastboot to get things working again.
What I've thought of is adding some device detection logic to the installer script and then having it run through the process of properly repatching the boot image after the main Magisk install finishes in order to get things to work (as opposed to having a supplementary zip file work through that after an official build is flashed).
I forked the official Magisk repo a while ago and honestly forgot about it, but since v17 hit stable since then, I'm going to rebase those proposed changes against that version. No ETA on that as of yet - I've started back at college, so time is already kind of a rarity; in addition, given the age of the Kindles already (5+ years!), it may not be a thing to sustain long term. I still have my 8.9", so testing isn't an issue, but I don't expect Magisk running on these specific devices to function as expected (so more than likely SafetyNet will fall, probably Magisk Hide as well). I'm not 100% sure how it'll turn out, but these are pretty much going to be unofficial builds for as long as I/anyone else willing to run builds sees a benefit to doing so. When a build works to my satisfaction, I promise it'll go up on XDA.
Click to expand...
Click to collapse

Successfully patched the boot image and installed magisk 18 and installed some modules and they work

Trey n said:
Successfully patched the boot image and installed magisk 18 and installed some modules and they work
Click to expand...
Click to collapse
Great! Will you post the boot image? What modules have you tried? Is Wifi, Bluetooth, and LTE working?

kgiesselman said:
Great! Will you post the boot image? What modules have you tried? Is Wifi, Bluetooth, and LTE working?
Click to expand...
Click to collapse
took me a while but also finally got it all working. Thanks for this guide. It may help us in the 7, 8 and 10 tablets. I also note my Jem is currently on CM13

monster1612 said:
It's complicated. I recommend not doing this unless you're willing to follow it to the letter - when I get to creating the automated patcher, this won't be necessary.
Make backups!!
extract the boot.img from your ROM .zip, copy it to the device internal storage
install the Magisk Manager app, download the Magisk .zip and choose "patch boot image"; navigate to said boot image file
copy the modified image to a computer (preferably one running a Linux OS like Ubuntu)
download the boot_cert and u-boot.bin files from the official LineageOS/CM device repo; place these files in the same directory as the boot.img file
open a Linux terminal pointed to the same directory as the boot.img file
run for i in $(seq 1024); do echo -ne "\x00\x50\x7c\x80" >> stack.tmp; done to create the remaining file
run cat boot_cert patched_boot.img > boot.img (assuming the Magisk image produced is named patched_boot.img); this is the boot "signature"
run dd if=u-boot.img of=boot.img bs=8117072 seek=1 conv=notrunc to tag the second bootloader on
finally, run dd if=stack.tmp of=boot.img bs=6519488 seek=1 conv=notrunc to add the stack file; copy the new boot.img back to the kindle
reboot into recovery, flash the Magisk .zip to build the environment, but do NOT reboot yet
choose "Flash .img" within TWRP, select the boot.img, and select "Boot" to flash to the boot partition; reboot to system once complete
profit!
Click to expand...
Click to collapse
This works on the Kindle Fire HD 7 as well, just use the files from the Tate repository.

Devo7v said:
https://forum.xda-developers.com/showthread.php?t=2128848&p=75525760
I know it's not for the 8.9" but I was able to get my 7" working by repeating the procedure in step 5. Magisk messes up the kernel on the Kindle so all you have to do is reflash the kernel. You'll need a fastboot cable to get in fastboot mode though.
Take a look at the few posts before the one I linked to.
Click to expand...
Click to collapse
I also have the same issue, but I'm confused as to your referencing for Step 5, because the guide says specifically not to flash the freedom-boot image if you already have a custom ROM present. Can you reiterate on what to do, please, or can I ignore this warning?

BrianSamsungTab said:
I also have the same issue, but I'm confused as to your referencing for Step 5, because the guide says specifically not to flash the freedom-boot image if you already have a custom ROM present. Can you reiterate on what to do, please, or can I ignore this warning?
Click to expand...
Click to collapse
I reflashed the freedom-boot and got everything working properly. It's been a few months so I don't remember if i had to continue anything when it finally booted, but I do know that I didn't lose any data. I still don't know if you need to flash freedom-boot, but it works if you do.

a little late to the party but-
i recently made the mistake of installing magisk and it put the kindle in a bootloop. is there a way to push the stock boot.img with this method or is that too quick and dirty
any advice is appreciated. im tempted to just do a full wipe via the stock recovery but if theres a more surgical method id go for it. i also have a linux debian machine available.

[RECOVERY][Android 10/11][Stock/SODP][XZ2/C/P/3] TWRP 3.4.0-0 [UNofficial]

The Sony Open Devices Project is always happy about volunteers (coding, testing, etc)
Also mainlining your favorite snapdragon powered xperia device into the mainline kernel is possible and we will be glad to help you!
Official site
Unofficial site
Code:
#include <std_disclaimer.h>
/*
*
* We are not responsible for bricked devices, dead SD cards,
* thermonuclear war, or you getting fired because the alarm app failed. Please
* do some research if you have any concerns about features included in this ROM
* before flashing it! YOU are choosing to make these modifications, and if
* you point the finger at us for messing up your device, we will laugh at you.
*
*/
Team Win Recovery Project 3.x, or twrp3 for short, is a custom recovery built with ease of use and customization in mind. Its a fully touch driven user interface no more volume rocker or power buttons to mash. The GUI is also fully XML driven and completely theme-able. You can change just about every aspect of the look and feel.
FAQ:
Just wiping your phone in TWRP lead into an encrypted & not readable userdata in the stock system.
You need to open advanced wiping and check the entries data and internal storage.
Of course clear the dalvik, too.
If you backed up system and/or vendor partitions and you want to restore them, make sure that TWRP setting "Use rm -rf instead of formatting" is set!.
If after the usage of NewFlasher or the OTA Updater or something else, which installs stock firmware parts you get stuck into the TWRP or SONY Logo, you need again to disable the verification with the vbmeta.img file and its parameters in fastboot.
fastboot & adb
https://developer.sony.com/develop/open-devices/get-started/flash-tool/useful-key-combinations/
https://wiki.lineageos.org/adb_fastboot_guide.html
https://developer.android.com/studio/releases/platform-tools
Weird problems not easily to reproduce by other users require that you make sure, that you
Use the newest platform tools (adb & fastboot)
Downloaded the newest firmware via Xperifirm from XDA and installed the newest firmware via Newflasher from XDA
Newflasher from XDA
Xperifirm from XDA
Removing the stock bloat apps via titanium backup may result in a boot loop. Use a file explorer to remove them, disable them or try to use my unfinished bloat removal script at github.
Your phone reboots into recovery, instead of system? Maybe it crashed too often due to a wrong installation or whatever?
In TWRP:
Code:
cat /dev/block/bootdevice/by-name/misc
shows you the reason.
TWRP -> [WIPE] -> [Advanced Wipe] -> Tick only the misc -> [Swipe to Wipe]
Known Bugs:
The "fastboot boot twrp.img" doesn't work, if you use the hardware buttons to open the blue fastboot bootloader mode. Only use adb, twrp or the android system to reboot into blue fastboot bootloader mode or flash the twrp.img, boot the device, reboot into bootloader and flash the original boot.img back, before booting into twrp.
You can also "fastboot reboot bootloader" in the blue fastboot mode.
This is a bootloader bug, maybe it gets fixed with the stock Q bootloader.
If "fastboot boot twrp.img" results into a blackscreen and reboot of the phone, take a look if you modified the DTBO partition.
It requires a special TWRP for every modified kernel/dtbo until I find some way to unify the twrp kernel with the DTBO (if the bootloader supports it).
Bugtracker:
TWRP Bugtracker -> If you think the problem is in TWRP
My Bugtracker -> If you think the problem is in my implementation
Bugreport:
A bugreport needs your device name, dmesg, the /tmp/recovery.log and a way to reproduce the issue.
If possible use
Code:
logcat -b all
instead of just a dmesg.
If ADB is not working to provide logs
VOLUP+POWER for 1 Seconds -> Forced crash to create a pstore
Boot the normal system.
Give me the files in
Code:
/sys/fs/pstore/
If the TWRP is stuck at the TWRP logo, the chances are high, that the decryption didn't succeed.
To rescue a not responding phone:
VOLUP+POWER for 3 Seconds -> RESTART with one Vibration.
VOLUP+POWER for 20 Seconds -> SHUTDOWN with 3 Vibrations.
VOLUP+POWER+CAMERA for 30 Seconds -> HARDWARE SHUTDOWN by discharging a capacitor.
Thank you very much for your help, code contribution & testing! (Random order):
@dees_troy and his team of volunteers for the TWRP code
@dhacke thank your for providing a download server
Shame on me if I forgot someone after searching through the thread and my PM's!
And many thanks to the few donators!
A telegram group for technical SODP stuff:
https://t.me/xda_tv
XDA:DevDB Information
TWRP, ROM for the Xperia XZ2
Contributors
MartinX3, Sony
Source Code: https://github.com/MartinX3-AndroidDevelopment
ROM OS Version: Android 10
ROM Kernel: Linux 4.x
ROM Firmware Required: Newest recommended
Based On: AOSP
Version Information
Status: Stable
Current Stable Version: 3.4.0-0
Stable Release Date: 2020-06-13
Created 2020-03-29
Last Updated 2020-06-20

Download & Installation
Download:
https://androidfilehost.com/?w=devices&uid=11410963190603893035
https://www.dhsfileserver.de/ftp/martinx3/ Thank you @dhacke for the second download server
Installation:
Update to newest stock firmware before unlocking!!!
Unzip the *.gz files with https://7-zip.org/ or Linux.
Enter fastboot via software, not hardware buttons. See "Known Bugs".
fastboot --disable-verity --disable-verification flash vbmeta vbmeta.img
fastboot boot twrp.img
Advanced menu -> "Install recovery ramdisk" -> Choose twrp.img
Reboot into installed TWRP
Want Root? -> Magisk
(Only if your phone doesn't boot to system anymore) Advanced menu -> "Fix recovery bootloop"
(Optional; Security degradation; Only if you know what you're doing) Switch SELinux to permissive (with my permissive.zip)

News
02.03.2021
Reuploaded the SODP TWRP with a workaround for Android 11 compatibility.
Click to expand...
Click to collapse
15.06.2020
reuploaded the stock twrp for the xz2 premium with a completely fixed touch.
Click to expand...
Click to collapse
14.06.2020
reuploaded the stock twrp with a later touch kernel modules initialization.
Hopefully fixing the randomly happening not working touch.
Click to expand...
Click to collapse
13.06.2020
thanks to the fixes in 3.4.0 we got now a twrp with the following enhancements for stock and sodp:
- this twrp will work with future 10.0 roms, you don't need a new build matching the security patch level of your rom.
- you can install this twrp again with the buildin ramdisk patcher. Please follow the installation instructions.
Click to expand...
Click to collapse
11.06.2020
switch to twrp 3.4.0
sodp twrp 2020-06 security patch level
stock twrp 2020-05 security patch level for firmware 52.1.a.2.1
now both twrp should work without a rom being installed (empty system/vendor/oem partitions) and still be able to decrypt your userdata.
Also the stock twrp touch should now always work instead of playing russian roulette.
Click to expand...
Click to collapse
11.05.2020
reuploaded sodp twrp for 2020-05 security patch level.
It didn't boot with the newest aosp.
Click to expand...
Click to collapse
07.05.2020
sodp twrp for 2020-05 security patch level.
Click to expand...
Click to collapse
14.04.2020
removed stock twrp for firmware 52.1.a.0.672 until sony releases the kernel sources of the new security patch level.
Otherwise the keymaster won't decrypt the userdata for twrp and twrp gets stuck on the twrp logo.
Click to expand...
Click to collapse
13.04.2020
stock twrp for firmware 52.1.a.0.672
sodp twrp for 2020-04 security patch level
hopefully fixed the touch problems of the stock twrp
fixed the forced read only partition mountings of system, vendor, odm
Click to expand...
Click to collapse
31.03.2020
stock twrp for firmware 52.1.a.0.618
Click to expand...
Click to collapse
30.03.2020
sodp twrp for 2020-03 security patch level
installing in ramdisk (to make it persistent) is impossible at the moment, because it is a 9.0 twrp which makes its ramdisk incompatible to the rom.
Of course monthly twrp releases in sync with the current patch level need to be released or i would need to remove the userdata decryption completely.
The stock twrp will follow, after it became ready.
Click to expand...
Click to collapse

30.03.2020
SODP TWRP for 2020-03 security patch level
Installing in Ramdisk (to make it persistent) is impossible at the moment, because it is a 9.0 TWRP which makes its ramdisk incompatible to the ROM.
Of course monthly TWRP releases in sync with the current patch level need to be released or I would need to remove the userdata decryption completely.
The Stock TWRP will follow, after it became ready.
Click to expand...
Click to collapse
PS: AndroidFileHost blocked me for doing too many actions at the same time.
Maybe I can upload it there tomorrow.
Done

can't wait for possibility to install in ramdisk
but now there is a working recovery :highfive:

31.03.2020
Stock TWRP for firmware 52.1.A.0.618
Click to expand...
Click to collapse

xz3 twrp
MartinX3 said:
31.03.2020
Click to expand...
Click to collapse
xz3 twrp won't boot, sodp version is good but there's no install on ramdisk?

hafiidh said:
xz3 twrp won't boot, sodp version is good but there's no install on ramdisk?
Click to expand...
Click to collapse
You mean SODP TWRP works, but not the stock TWRP on your XZ3?
You are stuck on sony logo?
Or on TWRP logo?
I need a bugreport with "logcat -b all" via adb.
Install ramdisk doesn't work at the moment, because it is a TWRP 9.0 hack, since TWRP 10.0 is ready.
But I wrote it in the news
Edit:
Reworked the thread a bit for more clarification

First! Wonderful job!
---EDIT------
Everything works fine! So nice!

@MartinX3 Great job as always bro! :good:
I ran it on my XZ2 (stock 52.1.A.0.618).
Phone has booted to the twrp screen (till the unlock pattern), but the touch is not working (log).
I didn't start from scratch (fresh install), for the record.
Tia!

serajr said:
@MartinX3 Great job as always bro! :good:
I ran it on my XZ2 (stock 52.1.A.0.618).
Phone has booted to the twrp screen (till the unlock pattern), but the touch is not working (log).
I didn't start from scratch (fresh install), for the record.
Tia!
Click to expand...
Click to collapse
Thank you
Are you sure the touch doesn't work?
I tested this release on the same firmware on my XZ2 in stock.
This confuses me now.
You booted stock .618 before and you tried to deactivate & activate the display?
Here it worked right out of the box.
And if it happens again, could you execute "start preptouch"
And if that not works, could you execute "/sbin/preptouch.sh"?

Sjll said:
First! Wonderful job!
---EDIT------
Everything works fine! So nice!
Click to expand...
Click to collapse
Thank you

MartinX3 said:
[...] Are you sure the touch doesn't work?
And if it happens again, could you execute "start preptouch"
And if that not works, could you execute "/sbin/preptouch.sh"?
Click to expand...
Click to collapse
I am. I have tried at least three times so far. Tried turning off and on the screen, but the slide to unlock didn't work, too.
"start preptouch", no response, no working touch
"/sbin/preptouch.sh"...
Code:
insmod: failed to load /sbin/clearpad_rmi_dev.ko: File exists
insmod: failed to load /sbin/clearpad_core.ko: File exists
insmod: failed to load /sbin/clearpad_i2c.ko: File exists
akari:/sbin # ls -l
Tia again

serajr said:
I am. I have tried at least three times so far. Tried turning off and on the screen, but the slide to unlock didn't work, too.
"start preptouch", no response, no working touch
"/sbin/preptouch.sh"...
Code:
insmod: failed to load /sbin/clearpad_rmi_dev.ko: File exists
insmod: failed to load /sbin/clearpad_core.ko: File exists
insmod: failed to load /sbin/clearpad_i2c.ko: File exists
akari:/sbin # ls -l
Tia again
Click to expand...
Click to collapse
And the service menu of android should also say that you use the clearpad driver.
And I assume the 9.0 stock twrp did always touch fine?
The script say that the kernel modules for the touch driver got loaded and I assume the sys path in the script file got executed too, after loading the .ko files.
And you have a normal European firmware?
That's now a mystery for me why it works for me and others, but not in your phone
Especially if you have the clearpad touch hardware

MartinX3 said:
And the service menu of android should also say that you use the clearpad driver.
And I assume the 9.0 stock twrp did always touch fine?
The script say that the kernel modules for the touch driver got loaded and I assume the sys path in the script file got executed too, after loading the .ko files.
And you have a normal European firmware?
That's now a mystery for me why it works for me and others, but not in your phone
Especially if you have the clearpad touch hardware
Click to expand...
Click to collapse
I got the touch working after copying the three .ko libs to the ramdisk /sbin folder, and a small editing (below) in the permissive.sh (also removed "$touch_id" == "3" from preptouch.sh).
Code:
setenforce 0
insmod /sbin/clearpad_rmi_dev.ko
insmod /sbin/clearpad_core.ko
insmod /sbin/clearpad_i2c.ko
echo 1 > /sys/devices/virtual/input/clearpad/post_probe_start
I know this is an awful workaround, but maybe this give you some hint (or sets you more confusing yet )
I'm with the sony stock customized_br fw, as always!

serajr said:
I got the touch working after copying the three .ko libs to the ramdisk /sbin folder, and a small editing (below) in the permissive.sh (also removed "$touch_id" == "3" from preptouch.sh).
Code:
setenforce 0
insmod /sbin/clearpad_rmi_dev.ko
insmod /sbin/clearpad_core.ko
insmod /sbin/clearpad_i2c.ko
echo 1 > /sys/devices/virtual/input/clearpad/post_probe_start
I know this is an awful workaround, but maybe this give you some hint (or sets you more confusing yet )
I'm with the sony stock customized_br fw, as always!
Click to expand...
Click to collapse
Ah you have a XZ2C, not a XZ2?
Because the removed ID 3 is for the XZ2 with clearpad touch.
But the script did already copy the .ko files into your /sbin before and executed the same code, you did now manually.
And according to your logs the setenforce 0 was already executed earlier.
So you just did the same the script did.
Well, yes you confuse me more
Could you try to just execute "echo 1 > /sys/devices/virtual/input/clearpad/post_probe_start" if the touch doesn't work again?
I wonder if the initialization is too early on your device.
Then I could try to delay the preptouch service to a later stage of the device boot.

MartinX3 said:
Ah you have a XZ2C, not a XZ2?
Because the removed ID 3 is for the XZ2 with clearpad touch.
But the script did already copy the .ko files into your /sbin before and executed the same code, you did now manually.
And according to your logs the setenforce 0 was already executed earlier.
So you just did the same the script did.
Well, yes you confuse me more
Could you try to just execute "echo 1 > /sys/devices/virtual/input/clearpad/post_probe_start" if the touch doesn't work again?
I wonder if the initialization is too early on your device.
Then I could try to delay the preptouch service to a later stage of the device boot.
Click to expand...
Click to collapse
I have a regular XZ2 (H8216), and its touch_id is 3 (customized_br fw), so I removed the Id 3.
"Could you try to just execute..." I've already tried that, with no success.
"Then I could try to delay the preptouch..." Cool... As you could notice, it worked here with that awful way I did it, so why not?
:good:

serajr said:
I have a regular XZ2 (H8216), and its touch_id is 3 (customized_br fw), so I removed the Id 3.
"Could you try to just execute..." I've already tried that, with no success.
"Then I could try to delay the preptouch..." Cool... As you could notice, it worked here with that awful way I did it, so why not?
:good:
Click to expand...
Click to collapse
But you tried it also yourself later.
You executed the script again without success at a later stage.
And adding it to the permissive.sh is like executing the script, just earlier.
I assume `cat /sys/devices/dsi_panel_driver/panel_id` gives you the "3" as result?

MartinX3 said:
But you tried it also yourself later.
You executed the script again without success at a later stage.
And adding it to the permissive.sh is like executing the script, just earlier.
I assume `cat /sys/devices/dsi_panel_driver/panel_id` gives you the "3" as result?
Click to expand...
Click to collapse
I ran the scripts/codes manually in the trwp root shell (with the no working touch screen).
Yep, running the script earlier (permissive.sh) did the trick (*.ko files in the ramdisk).
And 3 is the output of the panel_id (that's why I removed the 3 from the preptouch.sh).
Edit: I've noticed this...
The permissions of the libs copied from vendor to sbin (preptouch), as you could see here
Code:
-rw------- 1 root root 1070630 1972-01-22 17:46 clearpad_core.ko
-rw------- 1 root root 251702 1972-01-22 17:46 clearpad_i2c.ko
-rw------- 1 root root 296934 1972-01-22 17:46 clearpad_rmi_dev.ko
Patched twrp
Code:
-rwxrwxrwx 1 root root 1070902 2020-04-02 21:48 clearpad_core.ko
-rwxrwxrwx 1 root root 251830 2020-04-02 21:48 clearpad_i2c.ko
-rwxrwxrwx 1 root root 297054 2020-04-02 21:48 clearpad_rmi_dev.ko
Edit 2: chmod on copied files did the trick (preptouch.sh) - but sometimes still not working (need to boot twice, three times...):
Code:
#XZ2 "3" XZ2C "4" Clearpad
if [[ "$touch_id" == "3" ]] || [[ "$touch_id" == "4" ]]; then
cp /v/lib/modules/clearpad_rmi_dev.ko /sbin/
cp /v/lib/modules/clearpad_core.ko /sbin/
cp /v/lib/modules/clearpad_i2c.ko /sbin/
[B]chmod 777 /sbin/clearpad_rmi_dev.ko
chmod 777 /sbin/clearpad_core.ko
chmod 777 /sbin/clearpad_i2c.ko[/B]
insmod /sbin/clearpad_rmi_dev.ko
insmod /sbin/clearpad_core.ko
insmod /sbin/clearpad_i2c.ko
echo 1 > /sys/devices/virtual/input/clearpad/post_probe_start
fi
Maybe you find out a more elegant way to handle with that!
Edit 3:
I also got it permanently installed on kernel (combo keys does work, too), it's a bit tricky I gotta say, but it works.
You know, twrp's ramdisk.cpio kernel hack (old install procedure).

Hi Martin,
just wanted to let you know that your latest version for Sony stock (0.618 fw) does also not work for me (touch does not respond). Even 'fastboot reboot bootloader' before 'fastboot boot twrp-xz2.img' doesn't change anything.
Device:
Xperia XZ2 (H8216) running Android 10 stock (52.1.A.0.618)
Customized CH
Clearpad Touch version: 3