Is there any way to unpack the backup files created by the
Code:
adb backup -all
command? (Working since Android 4.0)
It seems this is a file-based backup and unpacking (and repacking) would allow tweaking some internal databases and files without needing to root.
I would like to know this also
I believe the ".ab" format is in some kind of "tar" compressed archive.
bakakaka said:
I believe the ".ab" format is in some kind of "tar" compressed archive.
Click to expand...
Click to collapse
Doesn't look like it. At least it's neither plain tar nor one of tarred gzip, bzip2 or compress. Also, it's not zip or rar.
So what's left?
- The android backup supports encryption. This does not necessarily mean that the archiver has to support that, too, since you could always first archive and then encrypt or vice versa.
- You can include .apk-files in backups. This suggests, that the archiver does not need to be über-effective, like bzip2, since that would cost time and cycles and .apk-files are already compressed, IIRC.
I don't know how to make any sense of this but maybe someone else does?
If anyone is still interested, here's a small utility that does this:
Apparently I can't post links, so add 'http' in front to get the URLs:
github.com/nelenkov/android-backup-extractor
More details about the format and the tool here:
nelenkov.blogspot.jp/2012/06/unpacking-android-backups.html
How to extract android adb ab backup
You can do it that way, you need dd and openssl. You can do it from linux, mac or windows with cygwin:
Code:
dd if=mybackup.ab bs=24 skip=1 | openssl zlib -d > mybackup.tar
scandiun said:
You can do it that way, you need dd and openssl. You can do it from linux, mac or windows with cygwin:
Code:
dd if=mybackup.ab bs=24 skip=1 | openssl zlib -d > mybackup.tar
Click to expand...
Click to collapse
Note that this only works for unencrypted backups though.
kapitan_petko said:
Note that this only works for unencrypted backups though.
Click to expand...
Click to collapse
I see
And the other way around:
1. create adb backup header
2. convert tar archive and append to header
Code:
echo -e "ANDROID BACKUP\n1\n1\nnone" > backup.ab
openssl zlib -in backup.tar >> backup.ab
Doesn't work
scandiun said:
You can do it that way, you need dd and openssl. You can do it from linux, mac or windows with cygwin:
Code:
dd if=mybackup.ab bs=24 skip=1 | openssl zlib -d > mybackup.tar
Click to expand...
Click to collapse
Doesn't work for me. I get this error: openssl:Error: 'zlib' is an invalid command.
What did I do wrong? Do I need to update my openssl?
CyanogenLover said:
Doesn't work for me. I get this error: openssl:Error: 'zlib' is an invalid command.
What did I do wrong? Do I need to update my openssl?
Click to expand...
Click to collapse
Run
Code:
openssl list-cipher-commands
And see if zlib is at the bottom
Which operating system are you using?
Nope, zlib is missing. I'm using Ubuntu 14.04. What do I do now?
Are there any other methods to unpack an .ab file?
any chance we can deflate .ab file on the phone itself ?
I know that one can deflate zlib by adding zlib header to the data then pass it through gzip:
printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" | cat - filename | gzip -d
Click to expand...
Click to collapse
source: printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" | cat - body.zlib | gzip -dc
I even tried ( and failed, cause i'm getting 'corrupt data' error
printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" | cat - /sdcard/backup.ab > /sdcard/backup2.ab
dd if=/sdcard/backup2.ab bs=24 skip=1 | busybox gzip -d > wa.tar
Click to expand...
Click to collapse
is the .ab ( without encryption ) is compress in some special way such that this technique isn't working ?
EDIT - SUCCESS !!!!
bu backup PACKAGE_NAME > /sdcard/backup.ab
dd if=/sdcard/backup.ab bs=24 skip=1 > /sdcard/backup2.ab
printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" | cat - /sdcard/backup2.ab | busybox gzip -d > /sdcard/backup.tar
Click to expand...
Click to collapse
although gzip throws an error of 'gzip: corrupted data' - the ".ab" file is deflated into .tar, so that running
Code:
tar xvf <TARFILE> <FULL_INTERNAL_FILE_PATH> -C <WHERE_TO_EXTRACT_FILE>
extract the file I want !!!
EUREKA !!!!
DON'T YOU GET IT !?!?!
NOW IT IS POSSIBLE TO MAKE BACKUP SCRIPTS FOR APPS ON THE DEVICE ITSELF FOR FREE!
gps3dx said:
any chance we can deflate .ab file on the phone itself ?
I know that one can deflate zlib by adding zlib header to the data then pass it through gzip:
source: printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" | cat - body.zlib | gzip -dc
I even tried ( and failed, cause i'm getting 'corrupt data' error
is the .ab ( without encryption ) is compress in some special way such that this technique isn't working ?
EDIT - SUCCESS !!!!
although gzip throws an error of 'gzip: corrupted data' - the ".ab" file is deflated into .tar, so that running
Code:
tar xvf <TARFILE> <FULL_INTERNAL_FILE_PATH> -C <WHERE_TO_EXTRACT_FILE>
extract the file I want !!!
EUREKA !!!!
DON'T YOU GET IT !?!?!
NOW IT IS POSSIBLE TO MAKE BACKUP SCRIPTS FOR APPS ON THE DEVICE ITSELF FOR FREE!
Click to expand...
Click to collapse
Is root required?
---------- Post added at 03:38 PM ---------- Previous post was at 03:24 PM ----------
gps3dx said:
any chance we can deflate .ab file on the phone itself ?
I know that one can deflate zlib by adding zlib header to the data then pass it through gzip:
source: printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" | cat - body.zlib | gzip -dc
I even tried ( and failed, cause i'm getting 'corrupt data' error
is the .ab ( without encryption ) is compress in some special way such that this technique isn't working ?
EDIT - SUCCESS !!!!
although gzip throws an error of 'gzip: corrupted data' - the ".ab" file is deflated into .tar, so that running
Code:
tar xvf <TARFILE> <FULL_INTERNAL_FILE_PATH> -C <WHERE_TO_EXTRACT_FILE>
extract the file I want !!!
EUREKA !!!!
DON'T YOU GET IT !?!?!
NOW IT IS POSSIBLE TO MAKE BACKUP SCRIPTS FOR APPS ON THE DEVICE ITSELF FOR FREE!
Click to expand...
Click to collapse
Ok thanks but it probably requires root. Testing it in my Samsung S7 official stock ROM it generates a 0 bytes file. Note that not all apps allow adb backup, you have to perform backup only in those that allow it.
If I do "bu" on android terminal I get:
sh: resetreason: can't execute: Permission denied. Killed.
scandiun said:
Is root required?
Click to expand...
Click to collapse
bah... I think I gave my terminal emulator root perm, although I did not initiate any "su" command.
this gave some extra perm to the emulator...
so my answer is that root is required to run this command on the phone itself... but NO root is required is you run it through ADB from a PC.
so, yeah, no real special news here, but the fact that there is NO need to copy the backup to PC, because deflating zlib, tar extracting can be done on the phone itself.
which means in return, that NO special openSSL is needed to extract the WA key file.
gps3dx said:
bah... I think I gave my terminal emulator root perm, although I did not initiate any "su" command.
this gave some extra perm to the emulator...
so my answer is that root is required to run this command on the phone itself... but NO root is required is you run it through ADB from a PC.
so, yeah, no real special news here, but the fact that there is NO need to copy the backup to PC, because deflating zlib, tar extracting can be done on the phone itself.
which means in return, that NO special openSSL is needed to extract the WA key file.
Click to expand...
Click to collapse
Having root I would prefer to use Titanium backup or similar. The problem with adb backup is that not all apps are allowed to be backed up. There's a flag in AndroidManifest.xml
scandiun said:
Having root I would prefer to use Titanium backup or similar. The problem with adb backup is that not all apps are allowed to be backed up. There's a flag in AndroidManifest.xml
Click to expand...
Click to collapse
if you got root you don't need to use external backup, you have access directly to /data/data and can view/edit WA databases.
also, to backup WA's key, you MUST install old version of the app, then perform the "bu" backup.
( as OP's script does ).
scandiun said:
Having root I would prefer to use Titanium backup or similar. The problem with adb backup is that not all apps are allowed to be backed up. There's a flag in AndroidManifest.xml
Click to expand...
Click to collapse
Have a question, adb doesn't backup all kind of data, or doesn't backup all kind of apk?
I need some clarification please
Youms said:
Have a question, adb doesn't backup all kind of data, or doesn't backup all kind of apk?
I need some clarification please
Click to expand...
Click to collapse
It backups all apks provided that you use the -apk flag. For the data, it only backs up data for those applications that are allowed inside the manifest. with the app System Info for Android you can check the AndroidManifest of each app. Android does not tell you explicitly which apps will be backed up or not. iOS works much better, iTunes will backup all the data of your apps (the apps itself not, it have to be re-downloaded from the Apple Store, and if some app is removed from there you can't install it anymore).
Please, don't use dd bs=24, instead use dd ibs=24, which is a bit faster, but really fast is only
Code:
tail -c +25 backup.ab | zlib-flate -uncompress | tar -tf -
Related
Hey, can someone tell me how to create a script to run a terminal command. I looked up google but didn't find any good results.
Open a new file, and write:
Code:
#!/system/bin/sh
your_command
another_command
another_one_xD
Save the file with sh extension, and run your script as follows:
Code:
su #optional, if your script needs root access.
sh yourscript.sh
If you get an error, probably need to grant permissions to your script, so:
Code:
chmod +x yourscript.sh
And try again.
Granting su is
$ su
Correct?
And thanks for the reply :]
Sent from my XT862 using Tapatalk 2
Yep
When you set the permissions, you only have to do that once right?
Yes.
I'm trying to build a script to move pics and video from my internal storage to external storage. I have the script made and on my phone but when I run it, I get an error: "failed on /storage/sdcard0/DCIM/Camera/*.jpg" - Cross-device link . Any idea what that means?
mmurphmsu said:
I'm trying to build a script to move pics and video from my internal storage to external storage. I have the script made and on my phone but when I run it, I get an error: "failed on /storage/sdcard0/DCIM/Camera/*.jpg" - Cross-device link . Any idea what that means?
Click to expand...
Click to collapse
Move (mv) only move the directory entry, not the actual data. Because of this, you can not move files from one disk to another (e.g. from internal to sdcard), hence the cross device error. You'll have to copy and then delete the original.
kuisma said:
Move (mv) only move the directory entry, not the actual data. Because of this, you can not move files from one disk to another (e.g. from internal to sdcard), hence the cross device error. You'll have to copy and then delete the original.
Click to expand...
Click to collapse
That seems to have done the trick but I have found another issue. When going into multiple directories (/storage/sdcard0/DCIM/Camera) from the script or even the root directory, I get a No Such file or directory error. But I can CD into that directory and manually cp and rm the files. Any idea why that is happening?
mmurphmsu said:
That seems to have done the trick but I have found another issue. When going into multiple directories (/storage/sdcard0/DCIM/Camera) from the script or even the root directory, I get a No Such file or directory error. But I can CD into that directory and manually cp and rm the files. Any idea why that is happening?
Click to expand...
Click to collapse
You've simply done something wrong. Show us the script.
Here is the script I created:
#!/system/bin/sh
#
#
#
#
#Move Pictures from Internal Storage to External Storage media
cp /storage/sdcard0/DCIM/Camera/IMG* /storage/sdcard1/DCIM/Camera
rm /storage/sdcard0/DCIM/Camera/IMG*
#
#
#
#Move Videos from Internal Storage to External Storage media
cp /storage/sdcard0/DCIM/Camera/VID* /storage/sdcard1/DCIM/Camera
rm /storage/sdcard0/DCIM/Camera/VID*
I put the script in the root directory and run it from there. Its permissions are rwxrwxr
mmurphmsu said:
Here is the script I created:
#!/system/bin/sh
#
#
#
#
#Move Pictures from Internal Storage to External Storage media
cp /storage/sdcard0/DCIM/Camera/IMG* /storage/sdcard1/DCIM/Camera
rm /storage/sdcard0/DCIM/Camera/IMG*
#
#
#
#Move Videos from Internal Storage to External Storage media
cp /storage/sdcard0/DCIM/Camera/VID* /storage/sdcard1/DCIM/Camera
rm /storage/sdcard0/DCIM/Camera/VID*
I put the script in the root directory and run it from there. Its permissions are rwxrwxr
Click to expand...
Click to collapse
Does /storage/sdcard1/DCIM/Camera exists? Try add:
mkdir -p /storage/sdcard1/DCIM/Camera
... to the script before you begin the copy. This will create the destination(s) in case of non-existent.
Also, your cp and rm command will fail with this error if there aren't any files named both IMG* and VID* in the source directory.
kuisma said:
Does /storage/sdcard1/DCIM/Camera exists? Try add:
mkdir -p /storage/sdcard1/DCIM/Camera
... to the script before you begin the copy. This will create the destination(s) in case of non-existent.
Also, your script will fail with this error if there aren't any files named both IMG* and VID* in the source directory.
Click to expand...
Click to collapse
Yes it exists. I can navigate to the directory in Root Explorer. I tried running the script in Root Explorer but it didn't move the two pics i have in the /storage/sdcard0/DCIM/Camera directory to the /storage/sdcard1/DCIM/Camera directory.
mmurphmsu said:
Yes it exists. I can navigate to the directory in Root Explorer. I tried running the script in Root Explorer but it didn't move the two pics i have in the /storage/sdcard0/DCIM/Camera directory to the /storage/sdcard1/DCIM/Camera directory.
Click to expand...
Click to collapse
And you DO have files named IMG* and VID* in your source directory?
kuisma said:
And you DO have files named IMG* and VID* in your source directory?
Click to expand...
Click to collapse
Yes. I took to pictures and then tried running it and it still fails. I wonder if its not working cause I'm on CM10?
mmurphmsu said:
Yes. I took to pictures and then tried running it and it still fails. I wonder if its not working cause I'm on CM10?
Click to expand...
Click to collapse
You realise that even with pictures in the directory, you'll still get this error from the next operations you try executing on the videos? Still, the script should a) still copy the images, and b) not terminate due to this error anyway (i.e. consider it a warning).
But you say the images are not copied. Quite odd, I'd say.
Add a path to the script (export PATH=/system/bin:/system/xbin plus the places you need), add a test "echo Running" to make sure the script actually is launched by Root Explorer. And add +x permissions for all, even "other". You can also add:
echo /storage/sdcard0/DCIM/Camera/IMG*
... before the first copy. This will show you what files matching your expression.
I'll just manually move them. I was just hoping for an easier way until CM10 is enabled to save pictures to the external SD card.
mmurphmsu said:
I'll just manually move them.
Click to expand...
Click to collapse
Take this as an opportunity to learn more about script programming, I'd say.
Experience is what we get when we expected something else.
kuisma said:
Take this as an opportunity to learn more about script programming, I'd say.
Experience is what we get when we expected something else.
Click to expand...
Click to collapse
I do a little in UNIX at work that's why I tried this. I really want to learn how to program for actual Android apps. Just need to find something to help me get started, as I have no clue where to start.
Where save the file ?
Hello
I'm french and don't speak english very well ....
I try to do what is explain in this topic ( for tethering with a Sony Ericsson KYNO V - ics 4.0.4 with root access ....
I don't understand where to save the file with sh extension
What i want to run is : #!/system/bin/sh
iptables -P FORWARD ACCEPT
iptables -t nat -A POSTROUTING -o rmnet0 -j MASQUERADE
My script is :
su root
sh scripttether.sh
Thank you very much for your help
BR
RoberGalarga said:
Open a new file, and write:
Code:
#!/system/bin/sh
your_command
another_command
another_one_xD
Save the file with sh extension, and run your script as follows:
Code:
su #optional, if your script needs root access.
sh yourscript.sh
If you get an error, probably need to grant permissions to your script, so:
Code:
chmod +x yourscript.sh
And try again.
Click to expand...
Click to collapse
Maybe a noob question, but how do you guys split and repack SGS3 boot.img ?
The usual perl scripts don't seem to work with any S3 boot.img I came across (neither for the Galaxy Tab 7.7 boot.img's btw).
I keep getting this error :
Android Magic not found in boot.img. Giving Up.
Click to expand...
Click to collapse
Thanks for answering.
To unpack, you can do this:
Code:
abootimg -x boot.img && mkdir newramdisk && cd newramdisk && zcat ../initrd.img | cpio -i --no-absolute-filenames
Of course, that assumes you have abootimg installed. The above will split the zImage and the ramdisk from the boot.img and then proceed to extract the files from the ramdisk. Some ramdisks are not Gzip compressed so in that case use cat instead of zcat. Also: run that as root to make sure you don't mangle the files' permissions. I haven't tried repacking, though. ("find . -print | cpio -o -H newc | gzip > ../initrd.img" followed by "cd .. ; abootimg -u boot.img -r initrd.img" worked for my U8800pro, but I've had no need to try it with GS3 images.) I'd start by looking at the tools that come with the official Samsung source distribution and guides that tell you how to build a Samsung kernel.
Thanks a lot for this thorough answer
Trying this right now.
Couldn't find a specific Samsung kernel-related tutorial, though good idea to go take a look at Samsung's official kernel documentation.
qwerty12 said:
To unpack, you can do this:
Code:
abootimg -x boot.img && mkdir newramdisk && cd newramdisk && zcat ../initrd.img | cpio -i --no-absolute-filenames
Of course, that assumes you have abootimg installed. The above will split the zImage and the ramdisk from the boot.img and then proceed to extract the files from the ramdisk. Some ramdisks are not Gzip compressed so in that case use cat instead of zcat. Also: run that as root to make sure you don't mangle the files' permissions. I haven't tried repacking, though. ("find . -print | cpio -o -H newc | gzip > ../initrd.img" followed by "cd .. ; abootimg -u boot.img -r initrd.img" worked for my U8800pro, but I've had no need to try it with GS3 images.) I'd start by looking at the tools that come with the official Samsung source distribution and guides that tell you how to build a Samsung kernel.
Click to expand...
Click to collapse
Great!! Thanks for your information.
Here is what I use. Inside there are three binaries and two perl scripts,, copy the binaries into /usr/bin/ or you can add them in their own place and add that to the path. Then use this to help you use the files
Thanks for that too, ima try those scripts
Getting this error :
~$ perl unpack-bootimg.pl boot.img
could not find any embeded ramdisk images. Are you sure this is a full boot image?
Click to expand...
Click to collapse
Apparently, from what I have been reading, Samsung uses a different type of kernels than other manufacturers.
Although there's a huge number of custom samsung kernels out there. There might be a way^^
Here is the kernel i'm trying to edit if anyone wanna give a try at unpacking it for me.
That is true up until the S3 boot.img/kernel They have always used a zImage. Now Google has forced them to move over to EXT4 system and change the kernel format.
That file is only 2.88 mb's that is way too small to be a full kernel. Even for stock with no tweaks. That's why you are having an error.
This is the original boot.img from the CM9 for Galaxy Tab 7.7 update.zip
However i get the same error when trying to unpack S3 stock boot.img or even CM10 boot.img, although when i try the same scripts on my Xperia Play's kernels they unpack properly.
Good thing if Google made Samsung do kernels like others
Hi,
Did you manage to unpack/repack the SGS3 boot image? I'm trying to modify init.rc in an international SGS3 (i9300).
I've managed to unpack the boot image (from /dev/block/mmcblk0p5) as per qwerty12's command but how do I repack it?
Thanks!
Hey fellas!!!
This is my first real contribution to my xda friends .I happy to share with you guys the script to make ODIN flashable rom for ext4 devices.
First of all I would like to say that this script currently supports only International variants of sgs2,sgs3 and SG Note.This is because I own only Note (similar to s2) and my friend has s3 which I used to make s3 compatible.I can support any devices if you guys can give me the required data
So now into the topic
In order to work with the script,you need the following:
A PC/Laptop with linux installed either as dualboot or in vm
A rooted Phone which supported by the script.
A usb cable
And of course you need your brain
Apart from the hardware,you will also need:
A stock rom from which you need to take the following:
cache.img
modem.bin
hidden.img
and a custom kernel(zImage) of you choice.(recovery.img and boot.img in case of s3)
Now download the script and do the following:
Extract ODIN-script.tar anywhere.
Run first_time.sh (Needed Only for the first time)
Put the above collected .img's,.bin and/or zImage inside ODIN-script directory
Connect your phone to PC/Laptop with usb debugging enabled.
Run the script and follow the instructions
Your Rom will be in the same directory in few mins
Note1:The Rom obtained by the above is not a full wipe rom.If you need to make it full wipe(factory reset),you need aditional empty data.img(for note/s2) or userdata.img(s3).I will upload data.img for Note.It "may" work on s2 too.But I havent tested.so its on your own risk
S3 users,I'm sorry I can help you .Because I dont know the size of /data partition to make an empty image.
Note2:You can always flash the rom with .pit and repartition.I have always used .pit to flash my rom.But still you are at your own risk.
Disclaimer:Do this at your own risk!!!I'm not responsible if you brick your phone.(Though you cannot if you do everything correctly )
Download link and changelog on next post.
And finally if you like my project and if you think you need to support me for future updates on this,Please do donate .And do not copy/edit/alter this work without my permission!!!
Download Links and changelogs
Changelog for v1:
Code:
[LIST]
[*]Initial release
[/LIST];)
Changelog for v2:
Code:
[LIST]
[*]Cleaned the code and redone entirely
[*]Added support for preload for some devices
[*]bug fixes
[/LIST]
Thanks to:
1.As-I9000 for zlibs and first_time.sh
vijai2011 said:
Reserved
Click to expand...
Click to collapse
Reserved!!!
really awesome...going to try now and let you know the feedback...
Ok...Thats good...Just an update...Dont add cache.img for now...It takes forever in recovery to flash....Found the issue and working on a fix
vijai2011 said:
Ok...Thats good...Just an update...Dont add cache.img for now...It takes forever in recovery to flash....Found the issue and working on a fix
Click to expand...
Click to collapse
thank you and i was about to test and i will refrain from doing that till you release the fix!
grgsiocl said:
thank you and i was about to test and i will refrain from doing that till you release the fix!
Click to expand...
Click to collapse
Hey...You can try it without cache.img as long as you have csc within you /system
But still if you have added cache.img,it will boot but only with couple of reboots and a looooong time in recovery
vijai2011 said:
Hey fellas!!!
Note1:The Rom obtained by the above is not a full wipe rom.If you need to make it full wipe(factory reset),you need aditional empty data.img(for note/s2) or userdata.img(s3).I will upload data.img for Note.It "may" work on s2 too.But I havent tested.so its on your own risk
[/COLOR]
Click to expand...
Click to collapse
If you needed FULL WIPE DATA - FACTORY RESET, you need to put the image cache file with the command wipe data...
as i9000 said:
If you needed FULL WIPE DATA - FACTORY RESET, you need to put the image cache file with the command wipe data...
Click to expand...
Click to collapse
Nope...instead I have added a empty data.img It will wipe off data...But didn't make it for s3 because I have heard s3 has /data and /sdcard combined.So if I make one for s3,it will wipe sdcard too.But need a confirmation on this
And I have fixed recovery issue too in v2 and it should be fine now.
Finally had a look on your script too.It also looks good.good job.nice work .May be we two can work together to make a kitchen???
Sent from my GT-N7000 using xda app-developers app
I used your script as guiding for customizing my own odin rom.
- I untarred my firmware.tar.md5
- $ simg2img system.img.ext4 system.raw
- $ sudo mount -t ext4 -o loop system.raw /tmp
I didn't change a thing and then repacked it
- make_ext4fs -s -l 512M -a system newsystem.img.ext4 /tmp
- chown 1000:1000 newsystem.img.ext4
- packed everything with $tar -H ustar -cvf out.tar *.img.* fat.bin
- signed the package md5sum -t out.tar >> out.tar
- mv out.tar out.tar.md5
I flashed it, had no problems, but after the flashing i see "failed to mount /system (invalid argument)
And as my system partition doesn't get mounted, i cant startup my phone.
Does anyone have a clue on what i could be doing wrong?
1bymany said:
I used your script as guiding for customizing my own odin rom.
- I untarred my firmware.tar.md5
- $ simg2img system.img.ext4 system.raw
- $ sudo mount -t ext4 -o loop system.raw /tmp
I didn't change a thing and then repacked it
- make_ext4fs -s -l 512M -a system newsystem.img.ext4 /tmp
- chown 1000:1000 newsystem.img.ext4
- packed everything with $tar -H ustar -cvf out.tar *.img.* fat.bin
- signed the package md5sum -t out.tar >> out.tar
- mv out.tar out.tar.md5
I flashed it, had no problems, but after the flashing i see "failed to mount /system (invalid argument)
And as my system partition doesn't get mounted, i cant startup my phone.
Does anyone have a clue on what i could be doing wrong?
Click to expand...
Click to collapse
If you did exactly what you posted,then,
1. You should not have got system.img.ext4 but system.img and so simg2img is simg2img system.img system.img.ext
2./temp in mount command would be your temp filesystem where system files are stored.Use a relative path like TempDir in your PWD
3. Are you sure the make_ext4fs line is correct?You said you used my script as reference and I have never used that command at all.Its like:
Code:
sudo ./mkuserimg.sh -s TempDir factoryfs.img ext4 ./temp 893386752B
(in mb is also fine)
where the numbers indicate the size of system partition in Bytes.Be sure to cd to ext4_utils before executing this command and should always be run as sudo as the mount is done as sudo.And donot include .ext4 at the end.Use just img.Works better.
4.again...chown should be sudo.As the img is owned by root and so you cannot chown it with basic user permissions.Simple user hierarchy
5.You did a blunder here in tar command if you put the original system.img and the newly packed img in the same directory.As the command will include all .img's which we donot want.
vijai2011 said:
If you did exactly what you posted,then,
1. You should not have got system.img.ext4 but system.img and so simg2img is simg2img system.img system.img.ext
2./temp in mount command would be your temp filesystem where system files are stored.Use a relative path like TempDir in your PWD
3. Are you sure the make_ext4fs line is correct?You said you used my script as reference and I have never used that command at all.Its like:
Code:
sudo ./mkuserimg.sh -s TempDir factoryfs.img ext4 ./temp 893386752B
(in mb is also fine)
where the numbers indicate the size of system partition in Bytes.Be sure to cd to ext4_utils before executing this command and should always be run as sudo as the mount is done as sudo.And donot include .ext4 at the end.Use just img.Works better.
4.again...chown should be sudo.As the img is owned by root and so you cannot chown it with basic user permissions.Simple user hierarchy
5.You did a blunder here in tar command if you put the original system.img and the newly packed img in the same directory.As the command will include all .img's which we donot want.
Click to expand...
Click to collapse
First of all, thanks for the reply.
second:
1. My phone is not rooted yet, it's a GT-S6500D aka Samsung galaxy tab. So I used a firmware.tar.md5 (that i have flashed to test and works)
and in this firmware.tar.md5 i have following files: boot.img, cache.img.ext4, fat.bin, hidden.img.ext4, recovery.img and system.img.ext4
So the system.img.ext4 wasn't a typo and the system.raw is just an arbitrary name i have chosen.
2. I don't get the mountpoint. Is this the mountpoint on the phone? of on my computer?
3. I have analysed the mkuserimg.sh script, and it is just a wrapper that flips the argument order. I have no clue why they made it..
So i doubt my problem would be in this line, but I will try, as i don't know what else can go wrong
4. chown should indeed be sudo, no doubt about that I just didn't type it here
5. I skipped this step a bit in the explanation, but i made the sure the directory only concluded the necessary files, naming the original files from step 1 or modified files.
6. I don't know if this could help, but if i unpack for example system.img.ext4 and repack it again, the new file will be bigger then the original one.. I was thinking maybe Samsung adjusted some things, but maybe i need some more arguments in my make_ext4fs command..
grtz
Fixed it!
The problem was that i was using a size of 512M without knowing where it came from.
I flashed the original firmware back to the phone, did a
Code:
adb shell "df /system "
to know what size my img needed to be. In my case it was a total size of 492M. So i used following command to make my system.img.ext4
Code:
make_ext4fs -s -l 492M -a system system.img.ext4 /system
Now trying to pack my own kernel in the boot.img
thanks for the support!
1bymany said:
Fixed it!
The problem was that i was using a size of 512M without knowing where it came from.
I flashed the original firmware back to the phone, did a
Code:
adb shell "df /system "
to know what size my img needed to be. In my case it was a total size of 492M. So i used following command to make my system.img.ext4
Code:
make_ext4fs -s -l 492M -a system system.img.ext4 /system
Now trying to pack my own kernel in the boot.img
thanks for the support!
Click to expand...
Click to collapse
Sorry for the late response .got a little busy.Happy that you solved it .Just an hint,you can use -h to see the size better readable by humans(In M and G) in df command incase you find its in kb or B
Sent from my GT-N7000 using xda app-developers app
Updated to v2
vijai2011 said:
Updated to v2
Click to expand...
Click to collapse
thanks man
I am working on a tool to help make updating or switching ROMs easier. Now, before you say: "Just use Titanium." I am attempting an AiO solution. Anyway, the way it is currently set up is to use an sh script via AROMA to tar a backup. I want the script to be customizable so it calls upon a text file the user can edit, called "keep.txt". This can easily be done using the tar command, however, when I use it and the subsequent sh to restore the files, I get mixed result on full restoration.
Here is an excerpt from the backup script:
Code:
#REMOVE JUNK AND WHITESPACE
sed 's/#.*//' /sdcard/NU/keep.txt > /sdcard/NU/keep-tmp.txt
cat /sdcard/NU/keep-tmp.txt | sed '/^$/d;s/[[:blank:]]//g' > /sdcard/NU/keep-tmp2.txt
#TAR CUSTOM BACKUP
tar -cvpf /sdcard/NU/Backup/Custom.tar -T /sdcard/NU/keep-tmp2.txt
# MOVING TEMP FILES TO LOG FOLDER
mv /sdcard/NU/keep-tmp.txt /sdcard/NU/Logs/keep-tmp.txt
mv /sdcard/NU/keep-tmp2.txt /sdcard/NU/Logs/keep-tmp2.txt
The first line removes all commented lines while the second removes all whitespace to allow for a cleaner input.
The second line should tar the backup appropriately keeping permissions intact (if I'm not mistaken).
The file chunk is for cleanup. Should also be noted that all folders are made at the start of install by another sh file.
An example of keep.txt can be found here: keep.txt
Here is an excerpt from the restore script:
Code:
# DELETE OG FILES
for f in $(cat /sdcard/NU/Logs/keep-tmp2.txt) ; do
rm -r $f*
rm -f $f
done
# RESTORE
tar -xvpf /sdcard/NU/Backup/Custom.tar
I've recently begun erasing the existing folders just in case... But this does not solve my problem.
The second chunk obviously restores the files.
It has had some success, but it is hit and miss. I don't want to release it and have people losing precious data.
Any input is welcome! Thanks for reading! :highfive:
I'm used to install RemixOS manually on a HardDrive partition and using GRUB for multibooting with Ubuntu, Windows, etc. This time I found some trouble installing new Marshmallow based build. It seems like Jide started using Squashfs+lz4 compression on this build and shipped squashfs-tools with Ubuntu can't manage it.
For the ones who don't know what "manually installing" means:
1- Download RemixOS .zip
2- Extract .zip and mount ISO extracted
3- Copy all ISO contents to a new partition where RemixOS will be installed ~8GB or more
4- Uncompress system.sfs using unsquashfs tool (this will generate system.img)
5- Deleste system.sfs
6- Create "data" folder alongside the files extracted from ISO
7- Make new custom GRUB entry for RemixOS on the correct used partition
8- Boot RemixOS from multiboot GRUB
So the best and easy way to solve the unsquashing problem is to download squashfs-tools source and building it with lz4 support.
Here the steps:
1- We' are going to build squashfs with full support for every available compress method (lzo,lz4,lzma,gzip) so we need dev packages, just run:
Code:
sudo apt-get install liblzo2-dev liblzma-dev liblz4-dev
2- Create a working folder and download squashfs-tools source
3- Open terminal go to working folder and uncompress source:
Code:
tar -zxvf squashfs4.3.tar.gz
NOTE: if they release a new version you need to replace "4.3"
4- Go to source folder and open makefile for editing:
Code:
cd squashfs4.3/squashfs-tools
gedit Makefile
NOTE: if they release a new version you need to replace "4.3"
5- In te Makefile uncomment (erase "#") in the following lines:
#XZ_SUPPORT = 1
#LZO_SUPPORT = 1
#LZ4_SUPPORT = 1
#LZMA_XZ_SUPPORT = 1
6- Compile and Install:
Code:
make
sudo make install
If everything goes right you can now open a new terminal and unsquash every squashfs file you can find and of course new LZ4 compressed from RemixOS. I hope this result helpfull for anyone!
NOTE: If you are not interested in having some un/compress type you can avoid installing xxx-dev corresponding package and DO NOT comment corresponding line on Makefile.
what about on elementary Os?
lukss12 said:
I'm used to install RemixOS manually on a HardDrive partition and using GRUB for multibooting with Ubuntu, Windows, etc. This time I found some trouble installing new Marshmallow based build. It seems like Jide started using Squashfs+lz4 compression on this build and shipped squashfs-tools with Ubuntu can't manage it.
For the ones who don't know what "manually installing" means:
1- Download RemixOS .zip
2- Extract .zip and mount ISO extracted
3- Copy all ISO contents to a new partition where RemixOS will be installed ~8GB or more
4- Uncompress system.sfs using unsquashfs tool (this will generate system.img)
5- Deleste system.sfs
6- Create "data" folder alongside the files extracted from ISO
7- Make new custom GRUB entry for RemixOS on the correct used partition
8- Boot RemixOS from multiboot GRUB
So the best and easy way to solve the unsquashing problem is to download squashfs-tools source and building it with lz4 support.
Here the steps:
1- We' are going to build squashfs with full support for every available compress method (lzo,lz4,lzma,gzip) so we need dev packages, just run:
Code:
sudo apt-get install liblzo2-dev liblzma-dev liblz4-dev
2- Create a working folder and download squashfs-tools source
3- Open terminal go to working folder and uncompress source:
Code:
tar -zxvf squashfs4.3.tar.gz
NOTE: if they release a new version you need to replace "4.3"
4- Go to source folder and open makefile for editing:
Code:
cd squashfs4.3/squashfs-tools
gedit Makefile
NOTE: if they release a new version you need to replace "4.3"
5- In te Makefile uncomment (erase "#") in the following lines:
#XZ_SUPPORT = 1
#LZO_SUPPORT = 1
#LZ4_SUPPORT = 1
#LZMA_XZ_SUPPORT = 1
6- Compile and Install:
Code:
make
sudo make install
If everything goes right you can now open a new terminal and unsquash every squashfs file you can find and of course new LZ4 compressed from RemixOS. I hope this result helpfull for anyone!
NOTE: If you are not interested in having some un/compress type you can avoid installing xxx-dev corresponding package and DO NOT comment corresponding line on Makefile.
Click to expand...
Click to collapse
Could you upload the system.img?
kabelon said:
what about on elementary Os?
Click to expand...
Click to collapse
I have never used it, if you're asking if it will work to do the installation steps for RemixOS, well as it is Ubuntu based I think the answer probably is yes.
Orion116 said:
Could you upload the system.img?
Click to expand...
Click to collapse
I'm on a 1Mbit connection and system.img is 2,7GB so I'm not able to upload it. If you are on windows try to find squash-tools for windows with LZ4 support. If you're running Ubuntu do the steps I have mentioned it's easy an it takes no more than 15 minutes.
lukss12 said:
I'm on a 1Mbit connection and system.img is 2,7GB so I'm not able to upload it. If you are on windows try to find squash-tools for windows with LZ4 support. If you're running Ubuntu do the steps I have mentioned it's easy an it takes no more than 15 minutes.
Click to expand...
Click to collapse
The issue is I am on Manjaro, an arch based OS. I'll try something similar on manjaro
Orion116 said:
The issue is I am on Manjaro, an arch based OS. I'll try something similar on manjaro
Click to expand...
Click to collapse
Try the same thing I described but using pacman I think it will work. If you fail (but I doubt) I will upload the image for you. But please try to do it because it is a big effort for me to spend my poor connection a lot of hours uploading the image.
Edit: just focus on lz4 package
lukss12 said:
Try the same thing I described but using pacman I think it will work. If you fail (but I doubt) I will upload the image for you. But please try to do it because it is a big effort for me to spend my poor connection a lot of hours uploading the image.
Edit: just focus on lz4 package
Click to expand...
Click to collapse
Seems I didn't have to install the packages
---------- Post added at 11:22 PM ---------- Previous post was at 11:17 PM ----------
lukss12 said:
Try the same thing I described but using pacman I think it will work. If you fail (but I doubt) I will upload the image for you. But please try to do it because it is a big effort for me to spend my poor connection a lot of hours uploading the image.
Edit: just focus on lz4 package
Click to expand...
Click to collapse
I got a system.img :good: Now to remember how to set up the grub for it:cyclops:
Orion116 said:
Seems I didn't have to install the packages
---------- Post added at 11:22 PM ---------- Previous post was at 11:17 PM ----------
I got a system.img :good: Now to remember how to set up the grub for it:cyclops:
Click to expand...
Click to collapse
Great.
Add this to custom file in /etc/grub.d
Code:
menuentry "Remix-OS" {
set root='(hdx,y)'
linux kernel root=/dev/ram0 androidboot.hardware=remix_x86_64 androidboot.selinux=permissive quiet DATA=/data
initrd /android/initrd.img}
(hdx,y) makes reference to the partition where you are installing RemixOS, Generally it is (hd0,y) where 'y' is the number of the partition
This setup works if RemixOS files are on root folder of partition
Edit: I need to warn you that my PC hang on a black screen after Setup Wizard on first boot if you have this problem ask me for the workaround
I have it booting but no WiFi, rats. It worked in the last LP build.
lukss12 said:
I'm used to install RemixOS manually on a HardDrive partition and using GRUB for multibooting with Ubuntu, Windows, etc. This time I found some trouble installing new Marshmallow based build. It seems like Jide started using Squashfs+lz4 compression on this build and shipped squashfs-tools with Ubuntu can't manage it.
For the ones who don't know what "manually installing" means:
1- Download RemixOS .zip
2- Extract .zip and mount ISO extracted
3- Copy all ISO contents to a new partition where RemixOS will be installed ~8GB or more
4- Uncompress system.sfs using unsquashfs tool (this will generate system.img)
5- Deleste system.sfs
6- Create "data" folder alongside the files extracted from ISO
7- Make new custom GRUB entry for RemixOS on the correct used partition
8- Boot RemixOS from multiboot GRUB
Click to expand...
Click to collapse
You don't need to unpack the system.sfs if you want
to manually install this. i.e. you don't need #4 or #5; and can save ~1.5GB of space. On the other hand if you want system rw - then you need the ext4 .img [as well as modified initrd.img]; and they seem to have started using better grub configs by default; i.e. using search instead of setting root to a partition.
HypoTurtle said:
You don't need to unpack the system.sfs if you want
to manually install this. i.e. you don't need #4 or #5; and can save ~1.5GB of space. On the other hand if you want system rw - then you need the ext4 .img [as well as modified initrd.img]; and they seem to have started using better grub configs by default; i.e. using search instead of setting root to a partition.
Click to expand...
Click to collapse
I actual just copied the grub.cfg for resident mode from the iso proper and used grub customizer to add it to my triple booted laptop. To bad Boardcom wifi is shot though otherwise it is perfect.
And how to add it to GRUB menu?
gb_14 said:
And how to add it to GRUB menu?
Click to expand...
Click to collapse
You can copy grub.cfg from original ISO into your /boot directory like Orion16 did. Or just edit /etc/grub.d/custom file as I told him in a previous post and run "sudo update-grub"
HypoTurtle said:
You don't need to unpack the system.sfs if you want
to manually install this. i.e. you don't need #4 or #5; and can save ~1.5GB of space. On the other hand if you want system rw - then you need the ext4 .img [as well as modified initrd.img]; and they seem to have started using better grub configs by default; i.e. using search instead of setting root to a partition.
Click to expand...
Click to collapse
Yes androidx86 first test for system.sfs, then system.img and last for system folder. On androidx86 builds I was able to boot with system folder and it was so in first RemixOS builds but now it is not working anymore (RemixOS loops at boot). I think that having installed an OS means also having RW permission over system files but it's just an opinion.
About modified initrd.img you don't need to unpack system.sfs for this method. But currently there is no modified initrd.img for last RemixOS and I don't have the time to do it.
I NEED TO WARN EVERYONE: if you modify system files mounting system.img as RW future OTAs will refuse to install and you will need to replace your custom system with the original one if you want OTA update.
lukss12 said:
Yes androidx86 first test for system.sfs, then system.img and last for system folder. On androidx86 builds I was able to boot with system folder and it was so in first RemixOS builds but now it is not working anymore (RemixOS loops at boot). I think that having installed an OS means also having RW permission over system files but it's just an opinion.
About modified initrd.img you don't need to unpack system.sfs for this method. But currently there is no modified initrd.img for last RemixOS and I don't have the time to do it.
I NEED TO WARN EVERYONE: if you modify system files mounting system.img as RW future OTAs will refuse to install and you will need to replace your custom system with the original one if you want OTA update.
Click to expand...
Click to collapse
I have one posted here
But yea if you go the full way [if you're on ext4] and dump the system.img into a system folder; then you don't need to modify the initrd.img for rw.
I used #1 on Bash for Windows; [just wanted to check the uncompressed system file] so kudos for that
HypoTurtle said:
I have one posted here
But yea if you go the full way [if you're on ext4] and dump the system.img into a system folder; then you don't need to modify the initrd.img for rw.
I used #1 on Bash for Windows; [just wanted to check the uncompressed system file] so kudos for that
Click to expand...
Click to collapse
Good to have an initrd.img but is it RemixOS 3.x based?
About system dump on system folder, it was not working anymore with recent 2.x builds (I haven't tested it on 3.x).
Now RemixOS 3.x build comes with built in SU so I can remount RW system from Terminal or using ESFileExplorer (or similar one) and only need to have system.img for editing files within RemixOS. (I forgot I was using modified initrd.img to achieve this with system.img as you said you need to make system folder dump to edit files within RemixOS and without modified initrd.img)
But as I mentioned it will break OTA... So having an intrd.img is important.
Btw if you see anyone having trouble with new Jide Setup Wizard in 3.x, the fix is to delete /priv-app/JideSetupWizard from system.img or RemixOS will not pass welcome screen
(So no OTA for me until Jide fix this up for my PC)
lukss12 said:
Good to have an initrd.img but is it RemixOS 3.x based?
About system dump on system folder, it was not working anymore with recent 2.x builds (I haven't tested it on 3.x).
Now RemixOS 3.x build comes with built in SU so I can remount RW system from Terminal or using ESFileExplorer (or similar one) and only need to have system.img for editing files within RemixOS. But as I mentioned it will break OTA... So having an intrd.img is important.
Btw if you see anyone having trouble with new Jide Setup Wizard in 3.x, the fix is to delete /priv-app/JideSetupWizard from system.img or RemixOS will not pass welcome screen
(So no OTA for me until Jide fix this up for my PC)
Click to expand...
Click to collapse
Yea, that one is 3.x based. I have a simple script somewhere so that you could make your own [its a simple unpack; search for ro; replace with rw and repack].
And JSW - I just disabled the app [with pm disable com.jide.setupwizard] instead of removing it.
Still not sure how you are getting rw without a rw initrd.img; with RemixOS just because you have su doesn't mean you can remount rw afaik.
Code from stock initrd.img:
Code:
check_root()
{
local r
if [ "`dirname $1`" = "/dev" ]; then
[ -e $1 ] || return 1
blk=`basename $1`
[ ! -e /dev/block/$blk ] && ln $1 /dev/block
dev=/dev/block/$blk
r=$(ls /sys/block/$blk/removable /sys/block/*/$blk/../removable 2>/dev/null)
[ -n "$r" ] && r=$(cat $r) || r=0
else
dev=$1
fi
try_mount ro $dev /mnt || return 1
if [ -n "$iso" -a -e /mnt/$iso ]; then
mount --move /mnt /iso
mkdir /mnt/iso
mount -o loop /iso/$iso /mnt/iso
SRC=iso
elif [ ! -e /mnt/$SRC/ramdisk.img ]; then
return 1
fi
removable=$r
zcat /mnt/$SRC/ramdisk.img | cpio -id > /dev/null
[ -n "$SYSTEM" ] && blk=`basename $SYSTEM` || blk=
if [ -b "/dev/$blk" ]; then
[ ! -e /dev/block/$blk ] && ln /dev/$blk /dev/block
mount -o ro /dev/block/$blk system
elif [ -e /mnt/$SRC/system.sfs ]; then
mount -o loop /mnt/$SRC/system.sfs /sfs
mount -o loop,ro /sfs/system.img system
mount_system_dev_img_if_necessary
elif [ -e /mnt/$SRC/system.img ]; then
mount -o loop,ro /mnt/$SRC/system.img system
elif [ -d /mnt/$SRC/system ]; then
mount --bind /mnt/$SRC/system system
else
rm -rf *
return 1
fi
mkdir mnt
if [ -n "$DEBUG" ]; then
echo " found at $1"
fi
rm /sbin/mke2fs
hash -r
}
HypoTurtle said:
Yea, that one is 3.x based. I have a simple script somewhere so that you could make your own [its a simple unpack; search for ro; replace with rw and repack].
And JSW - I just disabled the app [with pm disable com.jide.setupwizard] instead of removing it.
Still not sure how you are getting rw without a rw initrd.img; with RemixOS just because you have su doesn't mean you can remount rw afaik.
Code from stock initrd.img:
Click to expand...
Click to collapse
You are right!!! I was using a modified initrd for having rw on 2.x and I didn't remember . Will try pm disable thanks :good:
lukss12 said:
You can copy grub.cfg from original ISO into your /boot directory like Orion16 did. Or just edit /etc/grub.d/custom file as I told him in a previous post and run "sudo update-grub"
Click to expand...
Click to collapse
Thanks