solid, consistent benchmarking - Android Q&A, Help & Troubleshooting

Hey guys
I'm recompiling a kernel for my Galaxy S5. I'm disabling some parts of kernel code, trying different toolchains and compile options
But, how do I know if any of if is being effective in getting more performance out from my phone?
I know there are many benchmark apps out there, but I've tested ~10 and they all give inconsistent results. I mean, they all look pretty and stylish, but who can trust in a 1-minute test, with varying results even when not changing anything on the phone?
Don't tell me benchmarks are BS, because you have decent benchmarks for Databases (I work with some of them), Desktop PCs (even Windows had a built-in one), memory, disks, TVs, etc... I'm used to test and compare "things" forever, but I'm used to solid and consistent results. Sadly in Android world people give too much attention to pretty graphics or hearty words like "Your phone is fantastic" (this is the type of "conclusion" some benchmarks apps out there gives... LOL)
So, how to test a Kernel's performance in Android? Do I have to learn Java and create my own...? I hate Java!

Related

Benchmarking Software plz...

Lets all benchmark our XDA (MDA etc) . . . .
Then lets see how we can make them quicker!
Options:
- Registry Hacks?
- Overclocking Software?
- Etc . . . .
Surely somebody here has to be thinking along the same lines as me?
After all if you use something a lot, you want it better....
Can we clock this baby to 300Mhz + . . . .
MADmanFREE
:twisted:
I'm fairly certain anything over around 225-230 would exceed the refresh time for the memory and cause a hard reset. If it is even possible to manipulate the clock in the XDA by software.
Can someone explain
There is a large community devoted to overclocking. I have to ask why.
Is it really worth compromising the stability (and possibly integrity) of the hardware just for a few extra percent of speed?
These chips were designed to run at a certain speed (with a small margin to maintain stability). These chip designers know what they're on about, why do so many people think they know better?
I have to admit, i used to overclock my 486's, and it caused me so much trouble. Also a large high street retailer I worked for at the time used to overclock PC's and sell them as a higher spec - the amount of support calls we used to get!!!
By all means, if you want to try it, do so, but I have no problem with the speed of my phone.
Interesting so far.
Will have to get a poll going.
Just to see how much installing the extra software slows these things down...
MmF
JS's ipaqclock works, but i do not recommend oveclocking....

[Q] v6 super charger no effect?

Hello, I recall a while ago that people were reporting the v6 supercharger was dramatically improving their device performance. I have tried it on my YP-G1CWY/XAA multiple times, but it seams to make zero difference in performance. Has anyone had better results, if so, please list your device specs, and v6 settings.
Maybe I'm wrong, but the script should work better in systems that have small amount of ram. I don't think it will make a difference on the players which doesn't run the phone part of android.
V6 Supercharger is just a lowmemkiller tweak - kind of pointless on devices with plenty of RAM.
Like it's name, it's "all show and no go". In general, the more hyped up something is, the more likely it is to be useless crap.

Benchmarks

What do benchmarks actually measure? I always make allkinds of tests to the phones i have just for fun and to compare to others i have understood that a bigger score is always better..?
Benchmark test with antutu shows for me
10372
I have a s3 16gb and using the latest stock ics..
Sent from my GT-I9300 using xda app-developers app
CPU tests i guess is based on really long loops and depends on time that loop take you get result. RAM tests I think write big file direct to memory and depend of time get you speed of RAM memory in MBps or GBps...
On Antutu I have about 14,9K-15K with Paranoid and Siayah kernel. On Quadrant i have about 5,7K
Sent from my GT-I9300 using xda app-developers app
e-peen.
Seriously, different benchmarks measure different types of system performance by subjecting different elements of the device to different stress and performance tests. CPU calculstions, frames per second, javascript performance - they're genuinely useful in some situations, such as for devs to test out different approaches and ideas to see if one works fundamentally better than the other, and if done properly it can be useful to compare raw performance of one device to another, but they've cottoned on in the wider community because some people like to benchmark their phones and post those benchmarks proudly.
It's all a bit ridiculous, especially if - indeed - they don't know what they're benchmarking, or they're using out of date and largely useless benchmarks like quadrsnt, or they're just doing it for the bragging rights. It's relatively easy to push your phone and have it score well in assorted benchmarking tools if you really want to, it's just unlikely to be genuinely useful in that state - for example, setting the CPU governor to 'performance' had the effect of boosting it up to the highest available clock speed, and of raping your battery life. These are mobile devixes, not gaming PCs, and are designed to operate accordingly.
Personally, I ignore people's benchmark posts, because if they're posting benchmarks they're either brsgging, complaining about something absurd - "Your kernel doesn't perform as well as someone else's when in this unrealistic benchmarking scenario! It may best optimised for daily use, but now I can't hold my precious benchmarking score above everyone else's heads!" or "This new software update sucks because I don't get as high as score in Wankermark Extreme! [twenty sad faces]."
Hope that helps.
Sent from my GT-I9300 using xda premium
onslaught86 said:
e-peen.
Seriously, different benchmarks measure different types of system performance by subjecting different elements of the device to different stress and performance tests. CPU calculstions, frames per second, javascript performance - they're genuinely useful in some situations, such as for devs to test out different approaches and ideas to see if one works fundamentally better than the other, and if done properly it can be useful to compare raw performance of one device to another, but they've cottoned on in the wider community because some people like to benchmark their phones and post those benchmarks proudly.
It's all a bit ridiculous, especially if - indeed - they don't know what they're benchmarking, or they're using out of date and largely useless benchmarks like quadrsnt, or they're just doing it for the bragging rights. It's relatively easy to push your phone and have it score well in assorted benchmarking tools if you really want to, it's just unlikely to be genuinely useful in that state - for example, setting the CPU governor to 'performance' had the effect of boosting it up to the highest available clock speed, and of raping your battery life. These are mobile devixes, not gaming PCs, and are designed to operate accordingly.
Personally, I ignore people's benchmark posts, because if they're posting benchmarks they're either brsgging, complaining about something absurd - "Your kernel doesn't perform as well as someone else's when in this unrealistic benchmarking scenario! It may best optimised for daily use, but now I can't hold my precious benchmarking score above everyone else's heads!" or "This new software update sucks because I don't get as high as score in Wankermark Extreme! [twenty sad faces]."
Hope that helps.
Sent from my GT-I9300 using xda premium
Click to expand...
Click to collapse
200% agree but it wont stop the what is your quadrant score posts .
jje
it wont stop the what is your quadrant score posts .
Click to expand...
Click to collapse
Anyone good with Photoshop? Create a graph that reads 999'999'999'999'999'999 for your phone of choice and paste it under other people's claim.
It's neither more useful nor more useless than what they do
Benchmarks can be useful for yourself to check very basic performance before and after making changes (rom, kernel etc)
Sent via TCP/IP
gsw5700 said:
Benchmarks can be useful for yourself to check very basic performance before and after making changes (rom, kernel etc)
Sent via TCP/IP
Click to expand...
Click to collapse
Possible but run ten benchmarks one after the other and you get different results .
But as a method of a basic test to see if firmware has brought something extra to your phone yes .
Two areas of concern for users are always one use how the phone responds scrolls etc and that's subjective .Two is battery life and none of these benchmarks actually run a standard test that could be used to give a basic battery consumption figure over say a two hour period . But even that's not a good indicator of individuals battery usage .
jje

Advantages for having higher benchmark score

Hi guys,
I read many post that high benchmark score is just for fun or something like that. What matter is your daily usage, if it is smooth and responsive enough, then you can ignore benchmark result.
I am not completely agree with that. For me, benchmark score shows how efficient your ROM + kernel works.
If you get low score and not feel laggy, it means that you use your phone light enough, so even poor score doesn't matter for you.
For ex, when you open an excel file with a small data and simple calculation, it won't make much different whether you get lower/higher score in benchmark, it will open instantly. But when you open large excel file with complex calculation, then you will see the different.
Another example is when you run 3d benchmark with Antutu, with some ROM + kernel, it got around 25-30fps, another got 50-55fps. For gaming or app that use high def video a lot, surely, this will make a different.
Now, this is my experiences
1) On some ROM + kernel, I got Antutu's score around 22-23K, OC to 2496MHz
With this ROM + kernel, for daily usage, I don't feel any lag
2) On another ROM + kernel, I got Antutu's score around 32-35K, without OC
Since it has better score, I set it UC to 1.7Mhz and UV 100mAV and don't feel lag at all
So with a better score, I can UC and UV without effecting performance and the result is better battery life and if needed for heavy app, I can set it back to normal clock/volt to boost performance.
Regarding of this, choosing a ROM + kernel is not only based on performance, and I really-really appreciate ROM and kernel developer, they give ROM, kernel and times for free, hat's off for them, so
don't ask me with ROM + kernel that give best result
And then when you change just the kernel, what do you do?
Stock kernel, stock rom.. Antutu = 33K
Other kernel, stock rom... Antutu = 29K
Both at completely default settings. They both feel the same.
According to your theory, I should be able to UC and UV the stock kernel more then the other kernel, because it has a higher benchmark. But in practice, I have brought the other kernel down to 1.7 with no change in feel for actual usage. Bringing the stock kernel down to the same settings, there is more lag (as much as 3 seconds) opening some programs. Especially things that use the internet.
So the kernel that has the lower score can be underclocked more then the ne with the higher score. So the results are the opposite of what was predicted.
Things like changing TCP congestion to westwood have an effect on the responsiveness of some programs that isn't tested in the benchmark programs.
benchmarks really depend on the kernel...
I also experimented with different kernels.
I think benchmarks are for bragging right. some manufacturers have been caught fudging their benchmark scores.
So for me its not really that important.........
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
souleman said:
Things like changing TCP congestion to westwood have an effect on the responsiveness of some programs that isn't tested in the benchmark programs.
Click to expand...
Click to collapse
Hi,
TCP congestion have nothing to do with "responsiveness of some programs" like you said, nothing like this (nor speed, smoothess) is improved when you change the TCP congestion...
It's only about network stuff... http://en.wikipedia.org/wiki/TCP_congestion-avoidance_algorithm.
So how the TCP congestion could affect the responsiveness of some programs???
For sure also nothing improved in benches like Antutu, Quadrant, etc... when you change the TCP congestion...
viking37 said:
Hi,
TCP congestion have nothing to do with "responsiveness of some programs" like you said, nothing like this (nor speed, smoothess) is improved when you change the TCP congestion... For sure also nothing improved in benches like Antutu, Quadrant, etc... when you change the TCP congestion...
It's only about network stuff... http://en.wikipedia.org/wiki/TCP_congestion-avoidance_algorithm.
So how the TCP congestion could affect the responsiveness of some programs???
Click to expand...
Click to collapse
How many programs on your phone use an internet connection? Like any messenger program, many games, etc etc.. How many sync to an online server? Like gmail, facebook, etc.
So changing the TCP congestion algorithm can help programs that use the Internet because they are able to access the online information better.
Ok.
You should have explained this in your first sentence, this is confusing
It's not what I (we) call "responsiveness" of programs, but maybe it's just me :angel:
I said that because I've even read from some users that TCP congestion improves speed and smoothness in general UI
Debate closed for me.
Speeds don't always = smoothness. Some roms are built for pure speed that take away from day to day memory pools to help push graphics.
Numbers are a good reference for overall phone speeds compared to others but take it with a grain of salt.
htcm7 said:
Hi guys,
I read many post that high benchmark score is just for fun or something like that. What matter is your daily usage, if it is smooth and responsive enough, then you can ignore benchmark result.
I am not completely agree with that. For me, benchmark score shows how efficient your ROM + kernel works.
If you get low score and not feel laggy, it means that you use your phone light enough, so even poor score doesn't matter for you.
For ex, when you open an excel file with a small data and simple calculation, it won't make much different whether you get lower/higher score in benchmark, it will open instantly. But when you open large excel file with complex calculation, then you will see the different.
Another example is when you run 3d benchmark with Antutu, with some ROM + kernel, it got around 25-30fps, another got 50-55fps. For gaming or app that use high def video a lot, surely, this will make a different.
Now, this is my experiences
1) On some ROM + kernel, I got Antutu's score around 22-23K, OC to 2496MHz
With this ROM + kernel, for daily usage, I don't feel any lag
2) On another ROM + kernel, I got Antutu's score around 32-35K, without OC
Since it has better score, I set it UC to 1.7Mhz and UV 100mAV and don't feel lag at all
So with a better score, I can UC and UV without effecting performance and the result is better battery life and if needed for heavy app, I can set it back to normal clock/volt to boost performance.
Regarding of this, choosing a ROM + kernel is not only based on performance, and I really-really appreciate ROM and kernel developer, they give ROM, kernel and times for free, hat's off for them, so
don't ask me with ROM + kernel that give best result
Click to expand...
Click to collapse
there is no any advantage .. this bench crap thing is just gimmick makes children happy when they have higher score than others..
Using your theory that means many high end android phones a year or two ago should have been much smoother than the iPhone which clearly was not the case
Sent from my LG-D802 using xda app-developers app
shadehh said:
Using your theory that means many high end android phones a year or two ago should have been much smoother than the iPhone which clearly was not the case
Sent from my LG-D802 using xda app-developers app
Click to expand...
Click to collapse
that is not theory ! that is my state !
u need to start read more 'bout benchmarks tho .. it is app (program) for stupid people naive and pathetic .. well smart people making money of stupid people
http://www.anandtech.com/show/7384/state-of-cheating-in-android-benchmarks
http://blog.gsmarena.com/almost-all-android-makers-cheat-on-benchmarks/
It's only my opinion (maybe read closely to get whole point)...
viking37 said:
Hi,
Instead of always say "benchmark means nothing", "benchmark are useless", "benchmark are not real life" (obviously), bla bla bla... Maybe the OP could know why his score are low or why there is too much difference?
Like for some other guys if almost everyone have, it's an example, about 35 000 in Antutu and one guy post because he have only 20 000, I think he could know why, what is wrong, etc..., right?
But no, each time there is always the same kind of "ready repplies" like above...
Benches can be useful if you know how to run them (and yes there is a manner to run a bench like CPU temp at the beginning, with data active or not, after reboot or not, etc...) and how you interprete them (the most important, at least for me), even Quadrant can be useful for a quick bench but it's just my opinion (ok Quadrant seems to have more variations than Antutu)... And i don't manipulate them (like with the FPS unlock, I mean the debug.egl... build.prop edit) after it's for fun too.
And if I tweak my device with VM, governor, i/o scheduler settings for example (and it is not manipulating, not for me, it's some real "tweaks" that you can judge too in real use, in general UI, user experience (improvement or not) and not only in benchs) I see in first in general UI and after I look in bench to see...
Yes don't get these benches as a scientist approach or the best accurate result or a result to garantie you a general user experience (bad or good), for example my main goal is about general UI (smoothess, speed, responsiveness) and after I play with benchs because it's always fun...
But the tendance, here on XDA, is to ban benches with always the same words and the same mind (because xx says benchs are useless, xy repeat the same thing without knowing anything or xx have a bad/low score so benchs are useless, etc, etc, etc...), it bothers me... (it's not directed to you in particular).
And in general the guys who post a result of bench post a screenshot of the overall score and nothing more... The most important is to know how you got this score (governor settings, rom, kernel, "tweaks", etc...) and the detailed score is also important.
A bench makes sense with explications and details on how and why you have this or that score.
I agree that a screenshot without information means nothing..., I'm the first to say that in different bench threads but people don't care and continue to post a screenshot only, it's absolutely ridiculous, but well...
It's like the battery stats or screen on time (and we have at least to thrads for that...), it's a kind of benchmark, but here people agree more with these (?) and same thing: 2 or 3 screenshots without any information (settings or usage) means nothing...
In preparation of car engine you run benchs (for the raw power) and you test also your car on street to see the comportement in general not only on a bench and also ther is some conditions to respect before run a bench like for a phone. It's the same thing for a device if you like tweak it, you tweak your device, you run bench to see the result, you use it in day to day use to see if there is an improvement or not (general UI, battery).
It's the same thing for a sportsman, the 100 m, marathon, etc... it's a kind of bench, you don't use it always in "real life" nor it suggests your real state but people don't shout against, right? But benches for phones, welcome to the drama... it's pathetic.
"It shows benchmark apps are no different than a random number generator...", false, I posted some Antutu scores in the benchmark thread (testing i/o scheduler) and I can say you I have accurate results (just keep in mind I was testing different i/o schedulers so the overall score varies a little) in 35 800-36100 range, nothing with 2/4 000 points difference. I can show you accurate results in Antutu with always the same settings like 36 000-36 100 range..., is it a random number generator? No, I don't think... You will always have a little variation, if it's a variation of 5 000 points then something is wrong (thermal throttling mostly, an app auto update, who knows?).
Things to retain: detail your scores, explain your setup and yes don't get the result as a condition on how your phone runs in day to day use, that being said I don't know where is there a problem with benchs on phone... and sometimes fun is nice...
Ok, end, to the OP it's probably due to thermal throttling, but you give very little information... so hard to say exactly but it seems you ran Antutu one after another?
You need to let your phone cool down between each run...
Sorry for this post but I had something to say... and yes I assume totally
Click to expand...
Click to collapse
I'll have one or two things to add but it will be even too long (even some kernel (Elite)devs uses benches to test some stuff, even some Elite devs who claims that benches are useless develops bench apps, etc...), and the general idea should be here I hope...
And stop insulting with "stupid people" please...
I have the LS 980 and just flashed OpTimuS DE 1.5 with Furnace kernel. I just ran Antutu for the first time and got 20396. The chart shows the LG G2 over 30k. So I ran a second time and got 9907. This isn't right.
The only benchmark you really need is you. Only you know what you want from your phone, how you use it and where and which part of it you need the most.
This benchmark pissing contest is more and more boring annoying with every passing day.

New to anything like this. May I ask a few questions?

I barely understand what this means, however I have discovered that by going through tutorials, that one could do things like improve sd card performance, improve battery life, and many other things. Of course this interests me, but as I said I am new to android and I barely understand what this really means. I have hacked a lot of gaming consoles in the past and jailbroken many iphones, but that is about it. Definitely very new to all of this.
My device is the UMIDIGI A3X. Would it be possible to flash stock Android 10 onto this device (once I learn how to do this safely, still have to learn this) and then simply follow tutorials on kernel tweaks for the desired changes I want, without really truly understanding this kind of programming on a real level? In another words just going through the steps for the desired performance changes I seek.
Or am I way out of my depth here?
IMO SD-card performance basically has nothing to do with Android OS itself - but of course you can change relevant Android's cache size setting what probably doesn't result in any measureable results. Also you've to distinguish SD-card's IO read / write speed what depends on both the type of data bus the Android device supports and the SD-card's speed-class ( 2 - 10 ).
And battery life doesn't depend on Android OS: it solely depends on how the device - read: its CPU/GPU - is used/stressed and - most important - whether battery is always correctly charged ( never less than 30%, never more than 80% ).

Categories

Resources