[Q] How to pipeline pipe contents from android phone via adb to my computer? - Android Q&A, Help & Troubleshooting

Suppose that I do the following command from windows:
Code:
adb shell cat /sdcard/myfile.zip > c:\file.zip
The result is that the two files don't match, even they have much different file size. Using 'adb pull' works but is not compatible with pipelining or fifos.
Any help?
EDITED: here is the solution:
Antonine said:
Right, I'm currently working on a project which involves imaging Android phones - specifically, the program I've written is mean to automate identifying the memory block containing the userdata and then using the following sequence of commands (using Android Debug Bridge) to copy it to the computer that the phone is connected to:
Code:
adb forward tcp:5555 tcp:5555
adb shell
busybox nc –l –p 5555 –e dd if=/dev/mtd/mtd[mtd block number]
This is done in command prompt and then you have to manually open another command prompt and type in:
Code:
adb forward tcp:5555 tcp:5555
nc 127.0.0.1 5555 | dd of=[chosen file name].bin -- progress
This copies the memory block over without any problems when done manually and even shows you in real time how much data has been transferred.
The problem I have is with automating the process in C#. Specifically, I can get every stage of it to work through using a process to write the commands to standard input and using a tcp listening port to receive the data. Apart from this crucial element of the process:
Code:
adb shell "busybox nc –l –p 5555 –e dd if=/dev/mtd/mtd[mtd block number]"
For some reason, no matter how I try to phrase it or do it, C# hates that command and won't execute it. I can manually type the exact same thing into the command prompt and it works fine but trying to do it in C# just leads to busybox acting as though I typed the command in incorrectly - this is what it gives me:
Code:
BusyBox v1.15.2 <2009-11-27 10:38:30 GMT> multi-call binary
Usage: nc [-in] [-wN] [-l] [-p Port] [-f FILENAME|PADDR PORT] [-e PROG]
Open a pipe to IP:port or file
Options:
-e PROG Run prog after connect
-i SEC Delay interval for lines sent
-w SEC Timeout for connect
-f FILE Use file <ala /dev/ttyS0> instead of network
-l Listen mode, for inbound connects
<use -l twice with -e for persistent server>
-p PORT Local port
The problem appears to be particularly with the use of bs=[bytes] but I've no idea why - and I've spent several hours searching all over the web for solutions.
So basically, I was wondering whether anyone else might have encountered this issue before and if anyone has any ideas to get around it? At the moment I'm using a script to send the commands to the command prompt but that's a kludge that I'd really prefer not to have to use.
Thanks in advance for any suggestions or comments and apologies if I've posted this in the wrong place.
Click to expand...
Click to collapse
zeppelinrox said:
Probably has to do with syntax.
In the first instance you don't have it in quotes.
Code:
busybox nc –l –p 5555 –e dd if=/dev/mtd/mtd[mtd block number]
Then you do have quotes the second time.
Code:
adb shell "busybox nc –l –p 5555 –e dd if=/dev/mtd/mtd[mtd block number]"
Sometimes, using a newer busybox helps too... 1.15.2 is kinda old
Click to expand...
Click to collapse
NOTE: you have to use unix paths with cygwin, not DOS ones. Use cygpath to convert:
Code:
cygpath -u "C:\your path\folder\archive"

---

Thanks man, you saved my life! I just deleted some pictures from my Nexus 7 and I had to find a way to dump the internal storage to my computer in order to use it with PhoroRec...

---

scandiun said:
Suppose that I do the following command from windows:
Code:
adb shell cat /sdcard/myfile.zip > c:\file.zip
The result is that the two files don't match, even they have much different file size. Using 'adb pull' works but is not compatible with pipelining or fifos.
Any help?
EDITED: here is the solution:
Click to expand...
Click to collapse
can't seem to get the -- progress part to work it kicks back error of invalid command
Code:
" adb forward tcp:5555 tcp:5555 "
" adb shell "
" su "
" busybox nc 127.0.0.1 5555 | dd of=C:\Users\PB-2\Desktop\GN-3_MJ5_Full.raw.img bs=4096 "
After inputting the last line I get:
: cannot open for write: Read-only file system
any Ideas? I don't understand how my pc desktop could be read only
Thanks in Advance

---

Phatboyj420 said:
can't seem to get the -- progress part to work it kicks back error of invalid command
Code:
" adb forward tcp:5555 tcp:5555 "
" adb shell "
" su "
" busybox nc 127.0.0.1 5555 | dd of=C:\Users\PB-2\Desktop\GN-3_MJ5_Full.raw.img bs=4096 "
After inputting the last line I get:
: cannot open for write: Read-only file system
any Ideas? I don't understand how my pc desktop could be read only
Thanks in Advance
Click to expand...
Click to collapse
You can't use DOS paths on cygwin, you have to use unix ones:
Code:
busybox nc 127.0.0.1 5555 | dd of=/cygdrive/c/Users/PB-2/Desktop/GN-3_MJ5_Full.raw.img bs=4096
You can convert any DOS path to unix with cygpath if you don't figure out directly (even on other drives):
Code:
cygpath -u "C:\Users\PB-2\Desktop\GN-3_MJ5_Full.raw.img"
Result:
Code:
/cygdrive/c/Users/PB-2/Desktop/GN-3_MJ5_Full.raw.img

---

scandiun said:
You can't use DOS paths on cygwin, you have to use unix ones:
Code:
busybox nc 127.0.0.1 5555 | dd of=/cygdrive/c/Users/PB-2/Desktop/GN-3_MJ5_Full.raw.img bs=4096
You can convert any DOS path to unix with cygpath if you don't figure out directly (even on other drives):
Code:
cygpath -u "C:\Users\PB-2\Desktop\GN-3_MJ5_Full.raw.img"
Result:
Code:
/cygdrive/c/Users/PB-2/Desktop/GN-3_MJ5_Full.raw.img
Click to expand...
Click to collapse
Awesome Thanks,
I'm assuming this is true in both cd & dd via cygwin?
I was originally trying to input the commands in DOS because your OP doesn't specify, or say to use cygwin.
I came to the realization of using cygwin after reading other guides.
I've been looking at some of your other guides I owe You many thanks as a Result

---

Phatboyj420 said:
Awesome Thanks,
I'm assuming this is true in both cd & dd via cygwin?
I was originally trying to input the commands in DOS because your OP doesn't specify, or say to use cygwin.
I came to the realization of using cygwin after reading other guides.
I've been looking at some of your other guides I owe You many thanks as a Result
Click to expand...
Click to collapse
Yes, since cygwin is to be compatible with unix scripts paths must be always in unix format.
You can use dos paths directly
Code:
cd `cygpath -u "C:\your path\folder\archive"`
I've updated the first post

---

scandiun said:
Yes, since cygwin is to be compatible with unix scripts paths must be always in unix format.
You can use dos paths directly
Code:
cd `cygpath -u "C:\your path\folder\archive"`
I've updated the first post
Click to expand...
Click to collapse
Awesome share Thanks
I still can't get this to work "for the life of me" I know, I now have cygwin setup right, as it works with your example here

---

Phatboyj420 said:
Awesome share Thanks
I still can't get this to work "for the life of me" I know, I now have cygwin setup right, as it works with your example here
Click to expand...
Click to collapse
Are you still getting "cannot open for write"?

---

scandiun said:
Are you still getting "cannot open for write"?
Click to expand...
Click to collapse
Nope you set me strait on that one
In the first window I do
Code:
" adb forward tcp:5555 tcp:5555 "
" adb shell "
" su "
" system/xbin/busybox nc –l –p 5555 –e dd if=/dev/block/mmcblk0 bs=4096 "
and get the usage response
Code:
BusyBox v1.22.1-Stericson (2014-01-25 17:27:18 CET) multi-call binary.
Usage: nc [-iN] [-wN] [-l] [-p PORT] [-f FILE|IPADDR PORT] [-e PROG]
Open a pipe to IP:PORT or FILE
-l Listen mode, for inbound connects
(use -ll with -e for persistent server)
-p PORT Local port
-w SEC Connect timeout
-i SEC Delay interval for lines sent
-f FILE Use file (ala /dev/ttyS0) instead of network
-e PROG Run PROG after connect
If I trade out the string from your other example here, " The only problem i'm having with your other example is that I can't specify the " bs=4096 " "
I can get through to the second window
Example
Code:
[email protected] ~
$ adb forward tcp:5555 tcp:5555
[email protected] ~
$ adb shell
[email protected]:/ $ su
su
[email protected]:/ # /system/xbin/busybox nc -l -p 5555 -e /system/xbin/busybox dd if=/dev/block/mmcblk0 bs=4096
555 -e /system/xbin/busybox dd if=/dev/block/mmcblk0 bs=4096 <
The second terminal then gives me an error at the last string
Output of second terminal
Code:
[email protected] ~
$ adb forward tcp:5555 tcp:5555
[email protected] ~
$ nc 127.0.0.1 5555 | dd of=/cygdrive/c/Users/PB-2/Desktop/GN-3_Tmp/mmcblk0.raw bs=4096 -- progress
dd: unrecognized operand `progress'
Try `dd --help' for more information.
If I run it with out, " -- progress " it runs but with-out the desired effect of the progress response
Code:
[email protected] ~
$ adb forward tcp:5555 tcp:5555
[email protected] ~
$ nc 127.0.0.1 5555 | dd of=/cygdrive/c/Users/PB-2/Desktop/GN-3_Tmp/mmcblk0.raw bs=4096
I'm at a loss I need to figure out how to run the pv and the dd of= in the same line i guess
or to some how combine
Code:
" nc 127.0.0.1 5555 | dd of=/cygdrive/c/Users/PB-2/Desktop/GN-3_Tmp/mmcblk0.raw bs=4096 "
&
Code:
" nc 127.0.0.1 5555 | pv -i 0.5 > /cygdrive/c/Users/PB-2/Desktop/GN-3_Tmp/mmcblk0.raw "
So as to achieve both the bs= & the Read-out notification
Thanks tremendously, for your assistance thus far.

---

That's right, some android implementations of dd lack advanced parameters. You can try either installing Busybox from Stericson or just skip the bs option.

---

Removed all the "bs=4096" from quotations in first post, since it's not needed at all.

---

I keep getting
write error: File too large, when trying to dump my /data partition to my external sd card.
The external card is exFat formatted, which is supposed to be able to handle larger than 4Gig files
here's my command
Code:
busybox dd if/dev/block/mmcblk0p29 of=/mnt/extSdCard/data.ext4.img
any ideas?
p.s. yes, I do have more than 4G's of software on my phone

---

kevp75 said:
I keep getting
write error: File too large, when trying to dump my /data partition to my external sd card.
The external card is exFat formatted, which is supposed to be able to handle larger than 4Gig files
here's my command
Code:
busybox dd if/dev/block/mmcblk0p29 of=/mnt/extSdCard/data.ext4.img
any ideas?
p.s. yes, I do have more than 4G's of software on my phone
Click to expand...
Click to collapse
Two ideas:
1 - Try from recovery
2 - If that doesn't work put the sd into a card reader and connect it as otg drive.

---

scandiun said:
Two ideas:
1 - Try from recovery
2 - If that doesn't work put the sd into a card reader and connect it as otg drive.
Click to expand...
Click to collapse
I can try from recovery sure, but how would #2 solve the issue? (don't know what otg drive means)

---

kevp75 said:
I can try from recovery sure, but how would #2 solve the issue? (don't know what otg drive means)
Click to expand...
Click to collapse
Search google images for otg

---

scandiun said:
Search google images for otg
Click to expand...
Click to collapse
didnt work. I get the same issue
Rockin' it from my Smartly GoldenEye 35 NF1 (muchas gracias:* @iB4STiD @loganfarrell @muniz_ri @Venom0642 @ted77usa @rebel1699* @iB4STiD) ~ 20GB free cloud https://copy.com?r=vtiraF
Check me out online @ http://kevin.pirnie.us

---

kevp75 said:
didnt work. I get the same issue
Click to expand...
Click to collapse
Format a pendrive in NTFS or HFS+ and connect it to your phone via OTG, if it's not detected directly there are some apps on play store.

---

scandiun said:
Format a pendrive in NTFS or HFS+ and connect it to your phone via OTG, if it's not detected directly there are some apps on play store.
Click to expand...
Click to collapse
NTFS worked. My data partition is over 8Gigs! But I was under the impression that exFat was able to support that... but I guess not...

---

kevp75 said:
NTFS worked. My data partition is over 8Gigs! But I was under the impression that exFat was able to support that... but I guess not...
Click to expand...
Click to collapse
Next time you can try copying some big data to the exfat from the computer itself, just to see if prompts some error.

---

scandiun said:
Next time you can try copying some big data to the exfat from the computer itself, just to see if prompts some error.
Click to expand...
Click to collapse
tried that too had a 9.7g dvd imaage i tried with and it failed at the 4g mark. ext4 should be the standard
Rockin' it from my Smartly GoldenEye 35 NF1 (muchas gracias:* @iB4STiD @loganfarrell @muniz_ri @Venom0642 @ted77usa @rebel1699* @iB4STiD) ~ 20GB free cloud https://copy.com?r=vtiraF
Check me out online @ http://kevin.pirnie.us