Goal of this little project is to dispel myths and hearsay and trying to assess the elusive performance of custom kernels for our beloved SGS I9000.
So far this has proven quite challenging as there is no single good benchmark on Android (yet):
a lot of people have been misled by ridiculous Quadrant scores: ridiculous because, with some small tweaks which do not affect real performance in any way, shape or form, it is possible to boost the Quadrant score by factor 3x.
You're free to believe that your SGS I9000 which scores 3000+ on Quadrant is faster than a SGS II, but then please leave this thread and move on.
some kernels may seem smooth with some games, and get high scores on some synthetic benchmark, yet the UI appears "laggy" and stutters a lot in comparison to other kernels which score lower on the same benchmark
some popular benchmarks give results with unacceptably low reproducibility, i.e. if you run them multiple times without changing a thing on your system, you get scores varying by 50% of more, in a completely random fashion
most popular benchmarks do not measure or take into account multitasking and CPU contention with other applications, yet on a typical usage one has background tasks such as the media scanner or synchronization which kick in often and unpredictably
So this will be mostly a work in progress, i'm testing several benchmarks and several kernels in multiple combinations, trying to analyze which benchmarks offer certain criteria which make them useful, namely:
Reproducibility of results: running the same tests multiple times, should result in a very small variance of the final score
Performance separation: benchmarks which are too "synthetic" and show only a dependency on clock speed are not useful to discriminate "fast" kernels from "slow" kernels
Performance representation: we all know when a kernel "looks" or "feels" fast or smooth. If a benchmarks shows you that a "laggy" kernel scores higher than a fast and responsive one, it's likely that the benchmark is not well designed
I'll work more on this thread explaining my (current) choice of tests and what they're good for.
But for now i'll just post a link to the summary table, and give a brief recommendation concerning popular ICS kernels; recommendation which i'll explain in the coming days.
Base ROM:
Slim ICS 2.8
(because is fast, smooth and has the least background stuff of all ICS ROMs which i tested)
Test Conditions:
Whenever possible, i tried to overclock the kernels to 1.2GHz which most / all phones should have no trouble achieving.
In case of Semaphore i had to use the bus / live overclock but it wasn't fully stable at 1.2GHz on my phone so i ran most of the tests at 1.14GHz.
Tested Kernels:
Stock Teamhacksung V17
Devil 1.1.6b BFS
Devil 1.1.6b CFS
Icy Glitch V14 b
Semaphore ICS 0.9.5b
Recommendation:
Devil 1.1.6b CFS, Icy Glitch V14b (with SmartassV2 and FIOPS), and Midnight ICS (with a tweaked Conservative) are trading blows for the fastest kernel.
At the time of testing, Midnight is slightly worse in terms of overclocking though, apparently due to different voltages, also it doesn't allow overclocking beyond 1.2GHz.
But what's interesting is that it achieves great performance while using a tweaked conservative governor.
Devil 1.1.6b BFS is good but obviously inferior to its CFS brother.
Semaphore has the lowest cache and memory latency in the multithreaded test, it also has impressive sd card read speed and in general appears super responsive, but it's a bit worse in 3D gaming and especially it lacks "true" overclocking, "live overclocking" changes the bus clock and is way more unstable, in fact on my phone i couldn't run it stable at 1.2GHz.
All kernels are significantly faster than the stock teamhacksung's kernel, so you have no excuses not to upgrade to one of the popular custom kernels!
ICS 4.0.4
Started testing Android ICS 4.0.4 kernels on Slim ICS 3.2.
All tested kernels are "huge mem" versions with 380+MB of available RAM, without breaking video playback or 720p recording.
Summary:
the stock kernel from Teamhacksung is now a very respectable performer, unless you plan to overclock probably you don't need to install one of the other kernels
Semaphore, Midnight and Devil are all very fast and smooth
Results table:
https://docs.google.com/spreadsheet/ccc?key=0AuBUEB4dGFSSdHIyN2VIeWU4QnhLOFpJejFPWDh5S1E
Res 1
One request for the kernel developers:
could you please post me what are your preferred / recommended settings in terms of Governor and I/O Scheduler?
Only one configuration per kernel please, as running these tests is rather time consuming.
Test Settings
So for anybody who wants to follow the same methodology as I used to test kernels, please pay attention that in some tests i didn't use stock settings, to try to improve the reproducibility of the results.
Before all test, i put the phone in flight mode, and disable all synch services.
Antutu: DB I/O and SD Write and Read have poor reproducibility. So i run these tests separately 5 times, and take the best scores.
RealPi: the number of iterations is increased by factor 10x i.e.: 100000 digits
MPAC: lots of customization here. Also be careful as it's not very stable and some settings will make it crash.
All tests: 8 threads (or 8 producer / consumer pairs)
CPU: 10000000 iterations, use case: integer (i'm considering to add logical too)
Memory: stock apart for nr of threads. Repeat the test 5 times and get best numbers
Cache: 40 iterations
Res 3
With this should be enough.
Judging from those results, CFS Devil looks really promising.
Semaphore live oc stability issues happen only on Slim ICS indeed. On ICSSGS I have perfect stability at 1.2 ghz. And performance is just great, paired with very good battery life.
GT-i9000 / ICSSGS 4.2 / Semaphore 0.9.0
A quick question: did you lock the max freq to eliminate the "governor" variable?
Because each kernel could have governor's tweaks that the other don't.
Based on what you posted here, the differences between Glitch and Devil is practically none.
I tested both and didn't feel any tangible difference, in the end, it comes down to the unique features of each kernel.
Overclocking bus vs adding an extra step aren't even slightly comparable. Maybe do tests not overclocked?
Also there is a new glitch build with 100% working bln.
Sent from my GT-I9000 using xda premium
+1 for tests without overclock. Majority of us, users do not overclock. Maybe a seperate test for overclocking could be nice , but comparisons should be done with stock speeds imho.
Thanks for the time and effort. We needed this.
Overclocking bus Vs adding an extra step isn't an apple to apple comparison, I agree.
However my goal was to use each kernel in the best possible way, and if some kernels have the possibility to use higher multipliers / extra frequency steps, that is an advantage for the user, compared to the kernels who only offer live overclock.
Don't get me wrong, i love Semaphore and i've been using it for a long time.
And i have no doubt that some users can get it stable with live overclock to 1.2GHz.
But that is the ceiling, while with other kernels even my phone can reach stable overclocks of 1.5GHz, and that is something to consider.
I chose as the basis for my tests an overclock of 1.2GHz because it's something which practically everybody can use, without massive battery drain, overheat or shortening the life of the device.
I'll try to add measurements at stock speeds for those who don't like to overclock.
cba1986 said:
A quick question: did you lock the max freq to eliminate the "governor" variable?
Because each kernel could have governor's tweaks that the other don't.
Click to expand...
Click to collapse
I didn't want to take the governor variable out.
Because, as you said, each kernel could use (and often does) governor tweaks which make the kernel "special" or different from the others, and that has to be taken into account in evaluating them.
Because nobody will use the phone locked at the maximum frequency.
So for me the governor and its tweaks is part of the user experience of a certain kernel, and a distinctive factor.
At the end, all kernels are coming from almost the same sources, so it's the little things which make the difference.
phzi said:
Also there is a new glitch build with 100% working bln.
Sent from my GT-I9000 using xda premium
Click to expand...
Click to collapse
That's great!
This test i run is not the "be all end all", it was just a recommendation at the time of writing.
Pipperox said:
Overclocking bus Vs adding an extra step isn't an apple to apple comparison, I agree.
However my goal was to use each kernel in the best possible way, and if some kernels have the possibility to use higher multipliers / extra frequency steps, that is an advantage for the user, compared to the kernels who only offer live overclock.
Click to expand...
Click to collapse
Agreed but, then again, benchmarks should be done at original CPU clock IMHO.
Otherwise, results are distorted.
HiKsFiles said:
Agreed but, then again, benchmarks should be done at original CPU clock IMHO.
Otherwise, results are distorted.
Click to expand...
Click to collapse
agreed. Especially since stock team hacksung seems to be clocked at 1GHz
what's the point of the comparison? Really?
As expected, there is no noticeable difference between all 1.2 GHz kernels.
It's not as if there was a real difference between them anyway.
zorxd said:
agreed. Especially since stock team hacksung seems to be clocked at 1GHz
what's the point of the comparison? Really?
As expected, there is no noticeable difference between all 1.2 GHz kernels.
It's not as if there was a real difference between them anyway.
Click to expand...
Click to collapse
That's not quite true.
If you look closer, you'll see that Devil CFS has quite a distinct advantage over all others in 3D tests.
The point of the comparison between stock hacksung @1.0GHz and the others, who can overclock, is to show what kind of benefit you get from switching to kernels which are overclock friendly.
Especially considering that you can't assume that a 20% clock speed increase will bring a 20% performance speedup across the board.
At last, i'd say that you may have "expected" that the kernels tested at 1.2 GHz don't have such a difference in performance.
But expectations have to be verified.
I tried to answer the questions:
On Devil's kernel, is BFS really better than CFS?
The "popular belief" is that BFS is faster than CFS.
According to my tests, CFS results faster instead.
Another question may be, what kernel gives you the best gaming performance.
If you pay attention to the An3D Bench XL, you'll see that Semaphore 0.9.5b, even overclocked a 1.2GHz, is significantly slower than Devil.
If i recall correctly Semaphore Author claimed that some kernel developers overclock GPU, and he didn't. Idk anything about it, but i recall something about it.
Is it possible to overclock only GPU, without overclocking CPU??
zipgenius said:
so you should benchmark them without setting anything. Average users don't overclock and don't change governor or scheduler: they flash the new kernel and stop.
Click to expand...
Click to collapse
I completely agree on benchmarking every kernel at the same frequency (stock 1Ghz max) but I think there are two different options for further benchmarking:
1) Benchmark kernels configured as similar as possible regarding CPU governor, IO scheduler, readahead -> comparable results for all kernels.
2) Benchmark kernels with default settings (only makes sense if all compared kernels are optimized for similar purpose like performance, does not make sense if a kernel does *not* focus on max. performance and uses e.g. Conservative CPU governor as default setting.
@Pipperox: Would it be possible to check my Mindnight-ICS dev version with your benchmark suite? I'd be really interested in the results as you use the same setup for all kernels (1.2Ghz would not be a problem, does not use LiveOC but old school 1/1.128/1.2Ghz OC).
Interesting thread... I never used devil's CFS version, always BFS. Will try CFS out now.
@Mialwe Where can we get your ics kernel?
mialwe said:
I completely agree on benchmarking every kernel at the same frequency (stock 1Ghz max) but I think there are two different options for further benchmarking:
1) Benchmark kernels configured as similar as possible regarding CPU governor, IO scheduler, readahead -> comparable results for all kernels.
2) Benchmark kernels with default settings (only makes sense if all compared kernels are optimized for similar purpose like performance, does not make sense if a kernel does *not* focus on max. performance and uses e.g. Conservative CPU governor as default setting.
@Pipperox: Would it be possible to check my Mindnight-ICS dev version with your benchmark suite? I'd be really interested in the results as you use the same setup for all kernels (1.2Ghz would not be a problem, does not use LiveOC but old school 1/1.128/1.2Ghz OC).
Click to expand...
Click to collapse
Sorry guys, i understand your logic but i do not fully agree with it.
I'm not comparing overclocked kernels with heavy tweaking of voltages and special settings with which they only work.
I did the "poor man"'s overclock, setting to 1.2GHz using NSTools, a setting where 95% of phones should have no problem working.
I think that if some kernels offer you this possibility while others do not, it is fair to use this "advantage" that they have over the other kernels.
Because a lot of users will have the possibility to do the same as i do, without esoteric knowledge and with just a couple of clicks in the menus.
That being said, "due to popular demand" i will also try to retest those kernels at 1.0GHz as soon as i get a bit of time.
BUT in my recommendations, i will also consider the overclocking capabilities.
@mialwe: sure, i'll give a run to your kernel as well!
mialwe said:
@Pipperox: Would it be possible to check my Mindnight-ICS dev version with your benchmark suite? I'd be really interested in the results as you use the same setup for all kernels (1.2Ghz would not be a problem, does not use LiveOC but old school 1/1.128/1.2Ghz OC).
Click to expand...
Click to collapse
Dude, sorry but i don't seem to find your ICS kernel anywhere.. can you provide a link?
Related
Hello!
So, I have a DHD with the Revolution HD ROM (2.0.8) and the Buzz Sense kernel. I have locked the CPU to min/max 1516MHz Perf. scaling and, upon running the AnTutu System Review 1.2("System Benchmark" under Market) I get a score of 1879.
Yet in the Rankings I see that the #2 device is a DHD with 2.2 @1516MHz with a whopping score of 2774!!!!!
So, how could I reach those heights? The only thing I can think of is the slow SD card that came with the mobile (read/write scores: 34/117), but other than that what could be holding me back?
Cheers
T
I have reached pretty high scores while testing, but then the overall rom smoothness has pretty much sucked. It is normal that synthetic benchmarks give lower results when you use a custom rom, the optimizations that ensure general smoothness and usability cause it. One big reason is CPU scheduling, in custom rom (custom kernel) cpu time is divided for each task in a different way, thus reducing the overall benchmark score.
jkoljo said:
I have reached pretty high scores while testing, but then the overall rom smoothness has pretty much sucked. It is normal that synthetic benchmarks give lower results when you use a custom rom, the optimizations that ensure general smoothness and usability cause it. One big reason is CPU scheduling, in custom rom (custom kernel) cpu time is divided for each task in a different way, thus reducing the overall benchmark score.
Click to expand...
Click to collapse
Aha... but then this means that this or the other ROM do not let a specific app (say, a heavy 3D game) take the phone to its full potential...
Incorrect, it is just the synthetic benchmark that suffers. Games will run fine, faster than stock.
Jkoljo's right.The overall system speed is much better,performance is better,but it sucks a little in benchmarks.If you are so desperate to see high benchmark scores(I was once too! )try kamma's 1.4 kernel.Quadrant 3200.Need I say more?
tolis626 said:
Jkoljo's right.The overall system speed is much better,performance is better,but it sucks a little in benchmarks.If you are so desperate to see high benchmark scores(I was once too! )try kamma's 1.4 kernel.Quadrant 3200.Need I say more?
Click to expand...
Click to collapse
Really? 3200? Wow. Just out of curiosity, what scores are you getting with other kernels?
Well, at the end of the day, it doesn't matter - I haven't seen an app or game that runs slow or choppy or whatever. It's just for the heck of it
krakout said:
Really? 3200? Wow. Just out of curiosity, what scores are you getting with other kernels?
Well, at the end of the day, it doesn't matter - I haven't seen an app or game that runs slow or choppy or whatever. It's just for the heck of it
Click to expand...
Click to collapse
Oh yeah,I know what tou mean!
Anyway,with Lee's and Apache's kernels I get about 2500-2600 in Quadrant.CPU clocked all the way up to 1.5 GHz of course!
Iv'e read many posts throughout these forums about ROMS and/or Kernels being fast. I think my phone is fast enough for me. For others that may not be the case, so...
- to what extent does speed affect stability if any?
- aside from the obvious overclocking what else would one do to speed up their phone?
- what would you use these speeds for?
- then again what is fast, scrolling speed, transitions, streaming, gaming, browsing.....?
- what methods are being used to gauge these speeds, fps, linpack, quadrant.....other, are they acurate?
- its been said that AOSP is faster than Sense if so how?
It would be nice to gain a better understanding of various opinions on speed. Maybe some screenies of your set-up or different processes taken.
- Thanks for the insight
?
I would like to know the same thing i see alot of asop roms hitting 2000+ on quadrant i can only get to 1700 .....any help?
If you're getting 1700+ on sense, you don't need help. AOSP will be slightly faster just b/c there's less to it than sense. You're not going to feel a difference between a ROM running 1700 and one running 2000+ as long as you setup is running smoothly. Speed is more about feel than numbers anyway, although the popular tests are linpack and quadrant.
If you just want high scores, run CM6.1 and Snap 7.6 Kernel w/ Turbo activated.
what does set cpu do if anyone uses it?
jalai said:
what does set cpu do if anyone uses it?
Click to expand...
Click to collapse
Setcpu is used for over/underclocking your evo, also with screen off profiles, it can help increase you battery life.
I'm on the latest CM7 build and would like to know the pros and cons of using each kernel. Hopefully someone can enlighten me.
I'm on Pershoot's right now, btw.
personally i like the pershoot kernels, they give u more frequencies to choose from... now this isnt really a big deal when u picking the overclock frequency (844Mhz vs 864Mhz vs 883Mhz isnt a big difference really), but given that the kernel can vary the frequencies in real-time means that the kernel has more options to chose from, also given a good algorithm that picks the frequencies it should result in better performance and less battery use (theoretically since the power required for 806MHz is less than 883Mhz).
Also pershoot kernels are UV, which should mean that the CPU is using less power to run at any given frequency (compared to running at that exact frequency with a non UV kernel).
Having said so, CM kernel gives u more options regarding the governors, under the original CM kernel i always picked the Conservative governor which gave me amazing battery life... this option is not available under the pershoot kernel and i believe pershoot recommends using the OnDemand option (not a 100% on this one though).
so in my opinion, and if am correct it's mostly a preference thing. I use the pershoot kernels.
let me know if am missing anything
awsrasool said:
personally i like the pershoot kernels, they give u more frequencies to choose from... now this isnt really a big deal when u picking the overclock frequency (844Mhz vs 864Mhz vs 883Mhz isnt a big difference really), but given that the kernel can vary the frequencies in real-time means that the kernel has more options to chose from, also given a good algorithm that picks the frequencies it should result in better performance and less battery use (theoretically since the power required for 806MHz is less than 883Mhz).
Also pershoot kernels are UV, which should mean that the CPU is using less power to run at any given frequency (compared to running at that exact frequency with a non UV kernel).
Having said so, CM kernel gives u more options regarding the governors, under the original CM kernel i always picked the Conservative governor which gave me amazing battery life... this option is not available under the pershoot kernel and i believe pershoot recommends using the OnDemand option (not a 100% on this one though).
so in my opinion, and if am correct it's mostly a preference thing. I use the pershoot kernels.
let me know if am missing anything
Click to expand...
Click to collapse
Thank you so much! Just the answer I was looking for! Very informative and straight-forward, thanks.
Looks like I'll be sticking to Pershoot's.
awsrasool said:
Having said so, CM kernel gives u more options regarding the governors, under the original CM kernel i always picked the Conservative governor which gave me amazing battery life... this option is not available under the pershoot kernel and i believe pershoot recommends using the OnDemand option (not a 100% on this one though).
Click to expand...
Click to collapse
You're correct. Pershoot uses the ondemand governor because he says it's more stable at the higher cpu clock cycles. Few issues with FC's, lockups, and reboots. Though I do love the interactive governor (still want the smartass gov) he is correct from what I've personally experienced. That may be due to small variances in the kernels though despite coming from the same source but who knows.
Pershoot also has a .zip for every kernel that's full of some /lib files to add extra functions to your device, most users probably won't find a use for them though.
KCRic said:
Pershoot also has a .zip for every kernel that's full of some /lib files to add extra functions to your device, most users probably won't find a use for them though.
Click to expand...
Click to collapse
Can you please explain these extra functions and how to apply them please?
Wouldn't mind having some new functions
Sent from my HTC Vision using XDA App
I've found I get better battery life with Pershoot's kernel compared to the stock CM kernel using the same clocks speeds and governor (245 Mhz-1017 Mhz, Ondemand). No statistics to throw up though.
Ok im looking at kernels and im not going to ask "whats the best?" but im really not understanding the difference.
What im looking for is a kernel thats stable (that seems like all of them), one that allows under-clock/volting (and any other batter saving tricks) and one that will work well with my rom (XenonHD rc3) as most of the kernels seem to be using anyrom i dont think this is an issue.
i have been using the stock kernel then tinys kernel but im wondering if Zen or Air are going to serve me better?
Here is the order im looking at things
Stability
battery
speed
cosmetics
From what i can tell the governors dont seem to matter much as long as there are a few available (performance, interactive, conservative, power-saver) and the schedulers are even less important as they can handle normal use just fine. SIO or no-op or CFQ all work just fine for me. never tried FIFO but it seems kinda restrictive when multitasking
So from a development standpoint could someone explain whats so different in TINY, ZEN, and AIR i would much appreciate your input. They all seem to start from google source, are the compiled different?
Ok so i am trying Zen and i like that the CPU can be clocked lower. but im still not sure about whats best for me. A comparrison chart would be grand but i have no idea what to compare
The major differences between kernels are what kernel version they're compiled from, what modules are compiled into the kernel, which I/O schedulers are included, and which CPU governors are included. Depending on what the kernel dev has included, the kernel tends to run better or worse on specific devices. Unfortunately, it tends to vary quite a bit even within a single device line.
Zen is the best one I've found yet for my device. Others swear by Franco, Air, Trinity, etc. It's really a matter of trial and error on a device-by-device basis.
Finally, your statement about governors and schedulers not being that important is a bit wrong, in my opinion. Schedulers are definitely the lesser of the two, but depending on your usage, you can get a little bit of an I/O performance increase by using the "right" scheduler. The same thing goes for governors. A properly tweaked governor can save a bit of battery and/or boost your performance. Just like the kernels themselves, though, it would vary device-by-device and based on the user's usage type.
I started one for g4 plus now for g5 plus .
Cosmic os 2.1 unofficial
Elemental x kernel over clocked
What benchmark program are you using?
username8611 said:
What benchmark program are you using?
Click to expand...
Click to collapse
Antutu
PureNexus using ElementalX stock CPU speeds and GPU governor, CFQ, custom CPU governor settings
Lineage OMS with ElementalX kernel stock CPU speed and governor. ZEN with custom readahead.
This is kind of useless, benchmark comparison means nothing if it is not on the same device with same set of apps installed.
Sent from my LG G5 using XDA Labs
suhridkhan said:
This is kind of useless, benchmark comparison means nothing if it is not on the same device with same set of apps installed.
Sent from my LG G5 using XDA Labs
Click to expand...
Click to collapse
That doesn't make any sense. Devices are manufactured to a certain tolerance and winning the "silicon lottery" doesn't make a device faster, it makes it more overclockable. Device to device, stock for stock, the difference should be at most a few thousand points from each other. It should be pretty obvious to kill all background apps and processes before benchmarking so apps installed don't matter either. If Facebook is too important to kill for 10 minutes then that person shouldn't worry about benchmarking.
Device to device are obviously going to vary. But a varience of 10k+ points is a pretty good indicator of one set up running slightly better than the other and it's interesting to compare what is the most optimized settings. I can play with my CPU governor all day and get repeatable results +/- 500 - 1000 points. Both me and my wife had a Nexus 5 and with identical settings we both benchmarked very similar. To say it is a useless test is ignorant. If people look at this as a pissing match to see who's "better" then yeah, I see this being a dumb and useless thread. But I think most people who do this want to know what settings, ROM, and kernel are best optimized for performance.
Edit: https://www.phonearena.com/phones/Motorola-Moto-G5-Plus_id10398/benchmarks
63,191
https://www.youtube.com/watch?v=345eKlssdH8
62,769
http://www.fonearena.com/blog/214719/moto-g5-plus-review.html
62,893
https://www.pcmag.com/review/352573/motorola-moto-g5-plus
63,845
http://www.guidingtech.com/65986/moto-g5-plus-vs-redmi-note-4/
62,896
5 different devices, all tested stock within right around 1,000 points of each other.
username8611 said:
That doesn't make any sense. Devices are manufactured to a certain tolerance and winning the "silicon lottery" doesn't make a device faster, it makes it more overclockable. Device to device, stock for stock, the difference should be at most a few thousand points from each other. It should be pretty obvious to kill all background apps and processes before benchmarking so apps installed don't matter either. If Facebook is too important to kill for 10 minutes then that person shouldn't worry about benchmarking.
Device to device are obviously going to vary. But a varience of 10k+ points is a pretty good indicator of one set up running slightly better than the other and it's interesting to compare what is the most optimized settings. I can play with my CPU governor all day and get repeatable results +/- 500 - 1000 points. Both me and my wife had a Nexus 5 and with identical settings we both benchmarked very similar. To say it is a useless test is ignorant. If people look at this as a pissing match to see who's "better" then yeah, I see this being a dumb and useless thread. But I think most people who do this want to know what settings, ROM, and kernel are best optimized for performance.
Edit: https://www.phonearena.com/phones/Motorola-Moto-G5-Plus_id10398/benchmarks
63,191
https://www.youtube.com/watch?v=345eKlssdH8
62,769
http://www.fonearena.com/blog/214719/moto-g5-plus-review.html
62,893
https://www.pcmag.com/review/352573/motorola-moto-g5-plus
63,845
http://www.guidingtech.com/65986/moto-g5-plus-vs-redmi-note-4/
62,896
5 different devices, all tested stock within right around 1,000 points of each other.
Click to expand...
Click to collapse
Thank you for taking the time to write a long response. But, I believe you may have just proved my point. I believe the test results of different roms should be well within 'around 1,000 points of each other'. Unless-
a. the rom is very poorly optimized - score would be lower.
b. the kernel is overclocked - score could be slightly higher.
c. user error (lots of background apps).
suhridkhan said:
Thank you for taking the time to write a long response. But, I believe you may have just proved my point. I believe the test results of different roms should be well within 'around 1,000 points of each other'. Unless-
a. the rom is very poorly optimized - score would be lower.
b. the kernel is overclocked - score could be slightly higher.
c. user error (lots of background apps).
Click to expand...
Click to collapse
I really don't know how else to explain this to you. OP got a lower score than me, yet is overclocked. So it stands to reason that either "a. the rom is very poorly optimized - score would be lower" or "b. the kernel is overclocked - score could be slightly higher" or "c. user error (lots of background apps)" is the reason for it. But wait, the performance should be slightly higher for an overclock except that it isn't. That's the whole reason to benchmark. Another possibility is that since I've heard ElementalX is currently having overclock issues, it may be reverting to its nominal frequency, which I believe is 1.4Ghz. How would this person have known that if not for comparing benchmarks? According to you, they can't compare to stock benchmarks because it's a different set of apps installed and a different ROM and in fact can't compare it to anyone because it's a different device, albeit the same model.
Benchmarks show performance differences, regardless of whether or not they are large enough to even notice on a day to day basis. It shows technical differences and if you think technical differences mean jack squat, then why are you even commenting in this thread? It's the same theory when you throw a car on a dyno. You're going to notice small differences between each run, but when you have two of the same model cars with the same engine, and one consistently puts out 30HP more than the other, there's probably a reason for it.
To reiterate what I said in my first reply, for people who want to compare optimization between different ROMs, kernels, and technical settings such as CPU governors and schedulers, benchmarking is not useless. Not in this method of testing and not across identical devices with different software. The baseline or "stock vs stock" comparison shows that the benchmark is measuring with an adequate amount of accuracy and that multiple devices in stock form are performing equally before being modified. Just because it doesn't mean anything to you doesn't mean that it means nothing at all.
I did some research and things like backround apps running in airplane mode scripts like lightning blade. all these things make a difference. I was running kernel over clocked in interactive mode with lightning script. If I set to performance my score was significantly higher I was hoping this would give users a better way to set up and optimize their device not to compare roms running same device. Yes at first I thought about that then realized it wouldn't make a lot of sense. Im hoping some of u guys will hop on board and help test kernel roms and other mods so maybe we can get the best out of our device thanks guys.