Testing Methodology - Testing

Hi,
I'm new to (app) development, just wondering but is there a standard methodology when it comes to testing apps? I have finished my first app but aside from using heavily I'm not sure how to structure a test.
Thanks in advance for any pointers.

What exactly do you mean with testing? They are different types of testing:
1.) Unit Tests - When you write a function you should better write a unit test (e.g. with JUnit) - but this is for all type of code, not only for apps
2.) Automated Tests - You can build automated tests with e.g. Selenium Appium (you can google it and fit tutorials)
3.) Alpha & Beta-Tests by users - The Google Play Store allows to publish APKs to Alpha & Beta-Tests where you can specify which users should be able to download & test the App. This allows you to share your APK with other people before making it available for all

There are different tools/approach followed by software provider for testing mobile apps based on their types like Native apps,Mobile web apps and Hybrid apps .
Any testing method you consider will have pros and cons associated with it, and you will come to know that there is not a single testing approach to completely satisfy. Therefore, you can consider such qa services that combines different testing options as a whole and provide the best overall results.
The types of testing that can be performed on mobile app are:
usability testing
compatibility testing
performance testing
security testing etc.
- Mobile Emulators can be used for mobile testing.
-For unit testing on iOS, tools like Kiwi and OCMock can be useful.
-For Android, tools like JUnit with Mockito, Robolectric etc are available.
Almost every software testing company now a days offers different mobile app testing services and also provide help through their blogs, whitepapers, news letters etc.

Related

TeamSpeak Systems [Europe]: Senior Android Developer

NOTE: European residency required as you will be working with our Germany-based dev team. English fluency required. German fluency is ideal but not required.
HOURS: Full time preferred, part time will also be considered.
JOB DESCRIPTION:
We are looking for an experienced Android developer to create a mobile version of our popular, cross-platform client which is primarily used for voice chat (VoIP). You will have the opportunity to look at sample code for an already existing version of this mobile app (former developer code) which utilizes a client lib for most of its operational, non-GUI features. Our intent, however, is for you to recreate this app with a more polished GUI look and feel, in addition to adding numerous missing features which will bring the app up to par with its iOS counterpart.
RESPONSIBILITIES:
- Review and understand our existing Windows/Mac/Linux client-server voice chat product, how it operates, and what sorts of features and functionality our existing users will expect for its Android version.
- Work with our development team to determine initial set of requirements, and contribute ideas for improving the Android app's usability and overall user experience.
- Write clean, modular code to implement the desired requirements with little supervision, and submit periodically to dev team for review via subversion.
- Engage in primary, core testing of the app although our development team will also conduct some testing and report bugs/issues back to you as well.
- Work with our support team to document and fix bugs with reasonable turnaround. You will need to setup, manage, and maintain your own bug-tracking software (eg - Mantis or similar). Our support team will require access to this system and assist in reporting bugs and issues for you to work on.
REQUIREMENTS:
- You must reside in Europe since you will be working with our Germany-based dev team.
- At least 4 years in mobile development experience, with 2 years in-depth experience in Android development.
- Demonstrated track record for developing and releasing Android applications. You will be asked for sample work and code.
- Strong understanding of Android OS, developing in ADS, interactive application development paradigms, memory management, network programming, audio playback and microphone hardware integration, concurrency and multi-threading.
DESIRED QUALITIES:
- Strong interpersonal and communication skills.
- Ability to work within a small collaborative team and have a great passion in producing quality products.
- Demonstrated experience in working with others to solve challenging technical problems related to performance and usability.
- Self-starter with the ability to assess and resolve complex technical problems.
COMPENSATION:
You will be compensated as follows:
- A fixed, one-time fee to be paid in 3 parts.
Part 1 to be paid once the developer agreement is signed.
Part 2 to be paid once the initial, documented and agreed upon set of requirements has been completed.
Part 3 to be paid upon release of the Android app to the public.
- Ongoing percentage-based rev share based on actual (net) income received from app sales.
- Details to be negotiated and determined prior to hire.
Applicants should apply by submitting their resume or inquiry to the Business Development department via TeamSpeak’s ticket system at
http://support.teamspeakusa.com
Thank you for your time and consideration.

[TOOL] AppThwack - Easily test apps on real devices we host

Hi, I'm Trent and I co-created AppThwack, a service for on-demand automated testing of Android apps on real devices. Basically, you upload an apk and a couple minutes later you get screenshots and logcat dumps from actual phones and tablets we host.
Beta Users Needed
We're currently looking for devs to join our private beta. You can message me or sign up on our website and I'll get a beta code to you shortly. We're trying to stress the system and make additions/adjustments so the service is as useful as possible for developers.
Current Features
Real phones and tablets: About 20 high- and low-end devices and we add a few more every week.
Fast: See results in real time. Full test runs on all devices takes a minute or two to complete.
Selectable default tests: Install, launch, UI Monkey, Cleanup
Configurable tests: For example, specify the number of UI Monkey events and seed the randomizer
JUnit including Robotium support via uploaded test packages
Screenshots in portrait and landscape on all devices
Logcat and filterable logcat viewer
High-level results sortable by device or test
Full stack-traces for any exception that occurs
Future
We're working on adding more test frameworks like monkeyrunner and more default tests, particularly performance tests that measure battery consumption, CPU usage, etc. We're also adding more data visualization and charting so it's easy to see what some of the gathered statistics mean.
We plan to launch soon, but the beta program will remain in effect even after that. The service will follow a freemium model. Again, the beta is free and we're going to keep it in place even after we eventually launch.
Inter-device automation
Our back-end supports device-to-device automation, so if you have an app or scenario that you'd like to test that involves multiple devices or interaction with other devices, even non-Android devices, let me know. We're looking for people to help us develop how this service will be exposed.
Edit: I can't post links, but if you search for "appthwack" you'll find it. Btw, I've apparently lurked since October, 2008. Yikes.
Just wondering will this support of testing apps that require root?
Also any chance we can see like a live pic of the device when it installs the app and opens it?
Sent from my VS910 4G using xda premium
motodroidfreak said:
Just wondering will this support of testing apps that require root?
Click to expand...
Click to collapse
Right now it does not and all of our phones are as close to stock as possible. I'll look into making it an option so we automatically root before your app installs and then un-root after the tests are complete. Root opens up some new possibilities, both good and bad, so I'll need to think about it.
motodroidfreak said:
Also any chance we can see like a live pic of the device when it installs the app and opens it?
Click to expand...
Click to collapse
Yes! The launch test takes a screenshot in both landscape and portrait. You can see all screenshots sorted by device by clicking "By Device" or "By Test" and clicking the "Screenshots" link in the blue box at the top.
Screenshots also show up in each launch test log so you can see the context as the shot was captured. Logcat dumps show up in the same place (Link from the blue box at the top will open a filterable and highlighted log viewer).
Alright thanks I'll try it out tonight
Sent from my VS910 4G using xda premium
Holy cow. Didn't realize such web service existed.
I just signed up and currently having a look around. Is it possible for me to join the beta? Thanks!
Is it possible for a "free" upgrade for my account? Heh just asking
EDIT : Created a new project. Then I'm stuck. The "Runs" tab is empty.
EDIT again : Oh.. uploading had error previously. Uploading again.
Very interesting project. Good luck to your team and I hope I can be a good beta tester
Realy interesting, for us, almost of our apps needs root access, so please think about adding root to your service
Test on my Sensation
Will test on my sensation
test
nullFactory said:
Hi, I'm Trent and I co-created AppThwack, a service for on-demand automated testing of Android apps on real devices. Basically, you upload an apk and a couple minutes later you get screenshots and logcat dumps from actual phones and tablets we host.
Beta Users Needed
We're currently looking for devs to join our private beta. You can message me or sign up on our website and I'll get a beta code to you shortly. We're trying to stress the system and make additions/adjustments so the service is as useful as possible for developers.
Current Features
Real phones and tablets: About 20 high- and low-end devices and we add a few more every week.
Fast: See results in real time. Full test runs on all devices takes a minute or two to complete.
Selectable default tests: Install, launch, UI Monkey, Cleanup
Configurable tests: For example, specify the number of UI Monkey events and seed the randomizer
JUnit including Robotium support via uploaded test packages
Screenshots in portrait and landscape on all devices
Logcat and filterable logcat viewer
High-level results sortable by device or test
Full stack-traces for any exception that occurs
Future
We're working on adding more test frameworks like monkeyrunner and more default tests, particularly performance tests that measure battery consumption, CPU usage, etc. We're also adding more data visualization and charting so it's easy to see what some of the gathered statistics mean.
We plan to launch soon, but the beta program will remain in effect even after that. The service will follow a freemium model. Again, the beta is free and we're going to keep it in place even after we eventually launch.
Inter-device automation
Our back-end supports device-to-device automation, so if you have an app or scenario that you'd like to test that involves multiple devices or interaction with other devices, even non-Android devices, let me know. We're looking for people to help us develop how this service will be exposed.
Edit: I can't post links, but if you search for "appthwack" you'll find it. Btw, I've apparently lurked since October, 2008. Yikes.
Click to expand...
Click to collapse
I would like to test the tool, can you share with me..
Tested
Tested the tool, seems too good..
Suggestion : In-case if you want to reach maximum number of developer. Allow developer to use has free.
IDEA : You can request developer to post about you're tool on there app page, website & play store... As you're giving the tool as free you will get enough number of people to view & use the tool.. if the developer agree then you will allow him to use the tool for free of cost..
As a developer am ready to use the tool & post about you in my app & other places too...
This is a great tool, just uploaded an apk of my app that's in my signature and it worked, with a couple of NullPointers from the Play Store's licence service. That shouldn't happen, and doesn't on any of my devices, so I suspect it's an issue on your end. Any idea why?
HTML:
java.lang.NullPointerException at com.google.android.vending.licensing.LicenseValidator.verify(LicenseValidator.java:99) at com.google.android.vending.licensing.LicenseChecker$ResultListener$2.run(LicenseChecker.java:228) at android.os.Handler.handleCallback(Handler.java:605) at android.os.Handler.dispatchMessage(Handler.java:92) at android.os.Looper.loop(Looper.java:137) at android.os.HandlerThread.run(HandlerThread.java:60)
HTML:
FATAL EXCEPTION: background thread java.lang.NullPointerException at com.google.android.vending.licensing.LicenseValidator.verify(LicenseValidator.java:99) at com.google.android.vending.licensing.LicenseChecker$ResultListener$2.run(LicenseChecker.java:228) at android.os.Handler.handleCallback(Handler.java:608) at android.os.Handler.dispatchMessage(Handler.java:92) at android.os.Looper.loop(Looper.java:156) at android.os.HandlerThread.run(HandlerThread.java:60)
Borland
We are using Silk Mobile for end to end applications testing. Do you ever used this tool?
What an interesting service! I'll look into this from work tomorrow.
Not having used this at all, the first things which do spring to mind are:
-streaming realtime logcat
-a (skype?) connection with live streaming video of the app running, so you can see layouts/animations etc.
Anyway, I'm going to check this out tomorrow!
Quinny899 said:
This is a great tool, just uploaded an apk of my app that's in my signature and it worked, with a couple of NullPointers from the Play Store's licence service. That shouldn't happen, and doesn't on any of my devices, so I suspect it's an issue on your end. Any idea why?
Click to expand...
Click to collapse
Thanks for checking out the service. My immediate guess it that this is caused by the absence of a default Play account. Many devices have no account as one of our supported frameworks, calabash, removes accounts upon cleanup after script completion. On the plus-side, if you were to write scripts you should be able to add a temporary account from the test itself.
Highly unlikely you'd hit this bug in a real world situation, but it is a bug nonetheless.
Really awesome service! Can't test it for the moment as it doesn't support root apps, but this is really a great concept :good:
Maybe you could build a superuser permissions manager which would grant root access but makes sure to keep /system mounted as read-only, this way no harm could be done to the devices and us root apps devs could use your awesome service.
I actually got quite a few ideas, you could delete the mount binary in /system/xbin and use it in an internal appthwack app's private data, so that it's the only app able to call this binary and thus to mount /system.
I'd definitely subscribe to AppThwack if it had root support.
If you want help with developing this kind of secure root environment for the testing, I'd gladly contribute.
EDIT : Strangely enough, I just tested it with my app (which asks for root in the launcher activity, so I really didn't expect it to work) and had 0 failures, 75 pass.
How comes ? Have you already added root support ?^^
Either way this is really cool, I'm going to spread the words and most likely subscribe a paid account :good:
Is there somewhere we can see pictures of your device lab? Gotta be one hell of a device museum you got over there^^
Androguide.fr said:
Really awesome service! Can't test it for the moment as it doesn't support root apps, but this is really a great concept :good:
Click to expand...
Click to collapse
Awesome, thanks for the kind words!
EDIT : Strangely enough, I just tested it with my app (which asks for root in the launcher activity, so I really didn't expect it to work) and had 0 failures, 75 pass.
How comes ? Have you already added root support ?^^
Click to expand...
Click to collapse
This is pretty interesting. The only two rooted devices are a couple running CM. I'll look into this further, and if you have any ideas I'd love to hear them as well.
Either way this is really cool, I'm going to spread the words and most likely subscribe a paid account :good:
Is there somewhere we can see pictures of your device lab? Gotta be one hell of a device museum you got over there^^
Click to expand...
Click to collapse
Sweet, I really appreciate it. As soon as I have the number of posts to do so I'll post a pic of the lab.
Some things can not be automated, like scanning a QR code or reading/writing to an NFC tag. Do you plan on adding "manual tests" for a fee?
This would be really great to test apps on specific hardware.
worldtiki said:
Some things can not be automated, like scanning a QR code or reading/writing to an NFC tag. Do you plan on adding "manual tests" for a fee?
This would be really great to test apps on specific hardware.
Click to expand...
Click to collapse
Thanks for the question! Our primary focus is on automation. There are existing test houses and services that will execute manual tests like those you describe, but of course because of the manual component they're slow and expensive.
We often push folks to break their testing down into more granular chunks. For instance, verify you can take a picture and deal with the image, even if it's not the QR code or whatever your app usually consumes. This will find problems with simply using the camera and resulting image location. Now, have a separate test that processes a photo of a QR code, but feed the image in as part of the test. This removes the camera component from the analysis part, meaning it's now possible to benchmark the image analysis algorithm on all devices.
With a combination of a service like ours where you test very, very quickly on tons of devices, you can now do some more UX/end-to-end tests on a handful of devices yourself. This hybrid approach is great for finding the vast majority of issues before release.
nice post
Realy interesting, for us
Awesome tool !
No Developer can test his/her app on many devices. But your tool ... A W E S O M E ! ... I Used it yesterday to test two of my new apps ... Found some error is my app on certain devices which i'd have never found without AppThwack .. :good: :good:

[App][plugin] in-app feedback for Windows 8 apps

I am working on a web service and thought it might be of interest to Windows 8 app developers out there, who might find the service useful and could even help testing it and offer advice on how to improve.
The product (codename Myelin, currently in alpha) brings powerful user feedback tools directly into your mobile apps. With just a couple of lines of code, you can integrate functionality that not only allows users to send comments directly to the dev, but also to track any replies and provide additional follow-up after the first submission. No private information (such as email address or account name) is ever shared, and no registration is required. It just works directly from the app.
Coming in the future are even more exciting features that make meaningful communication between the dev and the end user simpler and faster.
On the backend we have a feedback management portal that allows to monitor incoming feedback efficiently and manage any required follow-up in a bugtracking-like approach (think support tickets).
We have recently rolled out a client (== app plugin) for Windows 8 HTML apps, and would welcome devs willing to take it for a spin and give us feedback. BTW, XAML support is coming in the future; if you'd be interested, let me know and this work may move further up the priority list. XAML version for C#\VB Win8 apps is also available.
The service is currently free while it's in active development. While there are plans to eventually take it commercial, we will in any event be very accomodating to our early adopters.
You can read more at https://www.tfp0.com/s/windows8. If you're interested in learning more, reply here, PM, or just go ahead and sign up over at the website (we have plenty of spots available) to see what we have going there.
Below is a collage of various screens that the plugin introduces in the form of settings flyouts.
<= clickable
Since I've seen some offline interest in a XAML-based version, I wanted to note here that we did in fact roll out a version of the plugin for XAML.
In addition, both versions (HTML and XAML) are now available through NuGet as Timefork.DyneinXaml and Timefork.DyneinHtml .

Use Apache Cordova?

Hey everyone! In response to a recent suggestion by a colleague, I've determined Cordova may be the best course of action for my current project. The reason we determined this, is because what I'm developing is already a web-app, that is complete, in which I can make a responsive style for mobile browsers & use Cordova to use a webview to access the app. So I'm looking for advice from experienced developers to see if this would be a viable option.
Let me start by giving you a run-down of the project. The project is called SecurSend. It is a private messaging platform that allows the user to submit sensitive information, without the worry of being monitored or keylogged. The end-user will compose a message which will generate a unique URL to the content, the recipient will receive this URL, and when accessing the content, it is permanently destroyed from the server. I don't currently have the site live, or it'd be much easier to show you, but the premise is security.
There is no current mobile theme for the site, however, it wouldn't be too difficult to implement. For something like this, do you feel Cordova would be the best option? Also, if you have any advice on Cordova, I wouldn't mind!
AutonomousHC said:
project is called SecurSend. It is a private messaging platform that allows the user to submit sensitive information, without the worry of being monitored or keylogged. The
Click to expand...
Click to collapse
re: the app:
thinking some combo of xmpp + OTR and webrtc might do it
(OTR adds end-to end encryption to xmpp and as browser support for webrtc improves there's peer-to-peer udp possibilities in webrtc)
Re Cordova:
. not yet sure but from looking at some javascript-based apps I've seen done with it what I've seen of it looks like an easier place to start for installable web apps (curious too)
Yes cordova is great for this. The only thing Cordova is not great at is high performance games but that is changing soon.
wowbro said:
Yes cordova is great for this. The only thing Cordova is not great at is high performance games but that is changing soon.
Click to expand...
Click to collapse
This sounds like an interesting concept, but I have heard you don't get the full native experience with Cordova (the UI seems different than that of native applications).
Do buttons and dialogs look the same as native apps?
Can you, for example, create transparent action bars, or side bars?
Or, notifications or watch faces for Android Wear?
wowbro said:
Yes cordova is great for this. The only thing Cordova is not great at is high performance games but that is changing soon.
Click to expand...
Click to collapse
Alternatively, you could use Steroids.js which has most native features built-in -- even Native UI components, MAPS API, Camera API, etc.
Cordova can achieve this, but as stated before, you will suffer some performance issues (especially Android 4.4 and lower)
I believe that doing web apps is the best for trying new ideas on all major platforms. The effort is very low compared to native development and performance is sufficient.
Ionic Framework has great performance and is continuously improving.
Though I wouldn't recommend web apps for games I recently released an open source zombie game called zland (zland.io).
Maybe React Native could be interesting for you as well. You will get true native performance.
With React Native you can share a code base between platforms but have to rewrite platform dependent thinks.
They recently released it for Android so you can deploy to both iOS and Android.
I estimate the effort somewhere between web apps and native apps.

[Q] Are there any effective tools for testing "context events" in Android?

I am a PhD student currently conducting research in mobile applications testing. My research focuses on the development of tools for testing context-aware mobile applications.
I am trying to gather information regarding the ability of currently popular automated mobile applications testing tools to test for context events (i.e. events that are not directly as a result of user input/actions) such as changes in data connectivity, changes in battery level, changes in GPS location, etc.
Can anyone provide any information regarding automated testing tools that can test for this kind of events? And as a mobile developer, are you happy with the ability of current mobile applications testing tools to test for context events?
Thanks.

Categories

Resources