My Most Favorite iOS Automation Tools!

Automation! I am a huge huge fan of automation. Whether it’s Mobile or Desktop … whether you use open source technologies or in-house built frameworks. Does not really matter :) Being a big believer in automation as ultimate solution for “need for speed”, I love following 3 iOS automation tools.

I was super happy when Google open-sourced Earl-Grey! Let us look at that first...

EarlGrey: https://github.com/google/EarlGrey

Google has open sourced it, Internally Google has used this framework for some of it’s products like Youtube, Calendar or photos et al

This is a native iOS automation framework which enables you to write clear tests. This framework gets integrated with XCode’s test navigator to run your tests from XCode itself.

One of the key feature is it’s built-in synchronization mechanism. This mechanism makes your tests run fast and provides improved performance. The built-in synchronization makes the test scripts to wait for UI events to occur.  

Then features like Visibility checks, use like interactions enables framework to be more richer and scalable over any other framework.

This framework is very easy to set up, just follow the instructions accurately provided on GitHub.

UI Automation: http://tinyurl.com/gwj9ux3

This one is an instrument. The one you can use for automating your user interface tests through some written test-scripts. This is mostly utilized for functional testing and it is provided by Apple.

You write tests in Javascript which differs from Obj-C. Since not many folks using it and neither Apple keeps updating it, any problem occurs, you don’t get much help with automation.

We had tried using it in one of my previous companies for UI functional automation however looking at various difficulties we had switched to something better.

KeepItFunctional: https://github.com/kif-framework/KIF

This is an integration testing framework. It uses XCTest testing framework to write tests.

KIF is an open source by Square which allows you to write tests using Obj-C. KIF supports all functionalities like tap, swipe, drag, click etc etc The tests are usually executed against iPhone or iPad simulators and usually gets executed speedier.

KIF is easy to set-up, install it, turn on the Accessibility inspector and configure it. The KIFTestController is nothing but a test runner. And KIFTestScenarios stores all the steps to perform a test scenario.

KIF provides an ability to integrate with Jenkins. After every Dev-Check-In or Commit, you can have a smoke test suite, written using KIF, gets executed to ensure nothing obviously is broken. KIF allows you to support and configure continuous deployment for frequent launches of your commits.

Is Quality time-consuming entity?

Lets first look at….What’s Quality? Is it just identifying defects by expert test team? That’s not quality, that’s just a way to get your product to a viable condition so you can release to world. And I agree it could be time consuming process.

How can we avoid it.

For that let us first even understand what is the meaning of “achieve quality”. Quality has always been associated with business context. The product concept need to be discussed thoroughly. Chalk out any imperfection right that step than coming into development cycle.

At the development phase, let us adopt test-driven design concept. The more effective unit tests you have, the early you will identify imperfections/flaws. This will allow you to move faster rather than waiting for test team to come into picture and find defects.

Follow an agile/lean process. That means literally eliminate anything that’s not adding any value! Get rid of unnecessary meetings, and the documentations. Adapt to user acceptance and short stories. Also avoid what we “need in future” as requirements / UI keep ever changing then why to even waste time in “need for future”

Use an On-demand Cloud Infrastructure.  Cloud is everywhere and we all know what are the benefits of using the same. Maintenance hazard is first and foremost thing we can avoid with this model. Provides flexibility, ensure data integrity, increases efficiency et al

Set up Continuous Deployment. I totally understand getting continuous deployment could be time consuming if there are hurdles faced. However get it clear that it is one time investment but benefits are for life time! This gives you a flexibility to “Build frequently and launch frequently”. Basically it equips you to get a small piece of functionality or feature done with TDD approach and get it out.

Opt-In for Parallel Executions. You can set up multiple important platforms running at the same time. Have your tests seamlessly integrated to run against these multiple combinations (you care about and/or identified from Marketing analysis et al) simultaneously.

Go for Usability Sessions. These are the best sessions. Having real users looking at your product and running through it, will help with finding UI issues/ simplification issues right away. So you can have your real users test it to identify exploratory defects. All of this can be done directly on Prod. You can always use A/B test mechanism.

Frequent Alpha-Beta Launches. You can start conducting alpha releases within your team itself or with little larger audience which constitutes product owners during development process. Alpha launch does not mean you have to have a viable and stable product. Once you are close to getting a stable product, you can opt in for beta launches. These sessions will help identify things way early to keep you moving faster!

From An Idea to Minimum Viable Product!

While working in Silicon Valley learnt about the concept of MVP Stands for Minimum Viable Product.

Let’s look at what MVP means:

In product development, MVP is described as getting minimum features developed just enough to get trusted customer feedback quickly.  This equips product members and various stakeholders to make quick decisions about their awesome idea or concept.

Usually various start-ups built MVP to know more about adaptability and revenue that their concept might generate.

Here’s how to go about achieving MVP to launch successfully and quickly:

Idea – Have an idea in mind... Just pen it down. Penning down your idea or concept would help you draw vision and audience base. It will help eliminate unknowns and uncertainties. This will also help you get align with focus.

Be Intelligent – Now you have vision, focus, and concept. What’s the next step? Figure out the problem. What pain-point you are trying to solve? Is this really needed? Should it really be built? Answer yourself to build the confidence! As the confidence, the positivity and the believe in idea will take you miles in the run…

Priority - Outlining relevant and important aspect for MVP is key.  As initial step, first identify what exactly you need out of features from your concept. Get it prioritized based on market analysis and trend.

Develop – Once you know what you want to reach users quicker, building the concept into actuality becomes a critical step. As you are building solution towards identified problem, in the process, you are learning too. Ensure to have metrics/analytics in place. As measurement and leanings will shape up your startup’s engine.

Cloud – With AWS, Google upcoming Cloud Solution, Micrsoft Cloud Solution…world is moving towards cloud and eventually scalability! However there are many cloud orgs which can help you pay for what you use model. These will help save tons of money providing sustainability and scalability.

YouTube – Yes Youtube! You might have heard that a video worth of million dollar words! J Youtube allows you to advertise with them. So instead of getting a heavy duty demo for your product or way too many pages to advertise, get a video built. Craft it carefully and message it smartly. Videos go viral now a days and it would be like mouth-publicity for your product to reach millions in no time…

Duration – As you need to get your idea quickly in market, you are short on time. In such cases focus on Mobile as thats the new and hot industry which will help you run your marathon and not a sprint. Focus on getting something out quickly on iOS and Android. You can put website/ support for all browsers et al to rest for now and build the mobile version first. Also buidling mobile version first help getting a web version up and running down the line.

Conclusion - Getting a Minimum viable product is key for your startup's success. Getting it tested by real end users helps you get feedback right away to modify or revisit the idea. 

JASMINE - A Unit Testing Framework

Any small or medium or large App does require unit tests to verify quality of the code during development stage itself. And this provides flexibility to easily change or modify code without worrying about breaking any regressions.

Before even getting into let’s understand BDD S/W development process.  BDD as it stands for behavior-driven development emerged from Test-driven development. BDD comprises of techniques and principles of TDD to provide ease on collaboration.

Jasmine, a JS unit test BDD framework, for validating JS functions. Jasmine does not have any dependency on DOM elements nor depends on any other JS unit test framework. This comes with it’s own syntax to write efficient tests.

Here’s you can get code for Jasnmine:

https://github.com/jasmine/jasmine

Jasmine provides flexibility either to use Ruby or Python. Once the setup is done as per above given instructions, you can start developing unit tests. It is advisable to develop your unit tests first and then write code.

Angular framework has written by keeping testability in mind. Jasmine whole-heartedly support any app built using Angular just like Mocha or qunit etc

You want to write a unit test for a simple mathematics equation

Sample code:

Describe (‘Maths, function () {

//necessary

it(‘should add two numbers accurately’, function() {});

it(‘should subtracts two numbers accurately’, function() {});

});

describe() method describes test suite. It() describes spec.

For Angular’s simple controller you can write below unit test:

var myApp = angular.module('myApp',[]);  myApp.controller('GreetingController', ['$scope', function($scope) {   $scope.greeting = 'Hola!'; }]);

describe(‘GreetingsController Example’, function() {

var GreetingController,

scope;

beforeEach(inject(function ($rootScope, $controller) {

scope = $rootScope.$new();

.

.

.

.});

Now you have an example of basic unit test using Jasmine. You can run all your tests on every commit. These tests are very lightweight, and executes faster.

Achieving Continuous deployment makes it easy.

On Jenkins, you can install a command line with grunt plugin. Add the command line argument in your Jenkins config file.

You can see a sample output here:

All About World of Mobile!

Mobile in general is such niche area and glad that at LinkedIn, I am getting to explore as well as expand new horizon.

My team is focus on supporting Mobile native apps, hybrid app as well as mobile web software test cycles.

I may not be able to write more here but look forward to a blog from me on engineering.linkedin.com

Exploratory Testing - Innovative approach

Exploratory testing is an approach to software testing that is concisely described as simultaneous learning, test design and test execution. While the software is being tested, the tester learns things that together with experience and creativity generates new good tests to run. Exploratory testing is often thought of as a black box testing technique. Instead, those who have studied it consider it a test approach that can be applied to any test technique, at any stage in the development process. The key is not the test technique nor the item being tested or reviewed; the key is the cognitive engagement of the tester, and the tester's responsibility for managing his or her time. (Ref: Wikipedia)


In a resource crunch situation, especially, when the new features are coming in to huge extent, this innovative approach can make wonders! However appropriate planning, organizing and motivation is desirable to make this approach a success.


In short, the approach, involves, getting various stake holders like engineering team members, product managers, UX team members, customer field representatives together and perform an exploratory testing... on given feature or product. It could be simple web 2.0 product or it could be wireless device...does not matter.


As long as QA manager/lead outlines proper strategy....and a step by step representation of how to effectively perform exploratory testing or corner case testing... Once you have such step by step guide you can always give a preso to above stakeholders and get them up to speed with concept of exploratory testing. And you can pin point exact ways to make it effective and get best out of it!!


When I had conducted this approach, I had come up with 10 steps of effective ways.... think like an end user, act like an end user....relevant to the product that I wanted to get tested thoroughly,... Made sure to get buy in from VP/ Director initially by explaining them approach, impact and result! Rolled it out to engineering. Then got buy in from product manager(s). Rolled it out to them and then field representatives as they deal with customers and they know the issues or most common operations by customers.


Overall to avoid duplication of bugs, we have had everyone perform testing at the same time..no matter where they are located within USA :) And then set up IRC channel where constant communication of issues seen was going on to avoid duplication of bugs.


Result..we actually end up finding some of the corner case issues, high priority issues! We ended up adding almost 35+ test scenarios in our test repository and automating the same!!


This was very well received by my engineering team as well as product manager. Helped save a lot of QA time of performing black box - exploratory testing ..

Selenium wait()

Selenium wait() not defined ? What will happen ?

Selenium uses the browser to make requests and process the server response. In short, selenium asks for a page and server sends back the response.
The browser then constructs a DOM tree out of the response. Selenium then parses this DOM tree and looks for elements as per the test statements.

Short example
Step 1 --> @selenium.open("www.google.com")
Step 2 --> @selenium.wait()
Step 3--> @selenium.is_text_present("Google ...")

Here the request is Step 1.

Now assuming that the server is not loaded at all, the network latency is very minimal and the client machine has a very fast processor, the time from the first request to receiving the response will STILL be atleast a few miliseconds. So not considering the network latency and delayed response from the server, some time is definitely required for the browser to generate a DOM tree out of the response and then ready the page to be rendered.

If we do not allow for a wait period, however small it maybe due to ideal network and server load conditions, then selenium will jump from step 1 to step 3. At this point, the DOM tree is not ready and selenium will NOT find the text "Google ..." in the DOM tree. Thus, our test will fail.

For any test environments, the server response times could be large and selenium has to wait for the response to be completely consumed by the browser and build a DOM tree. Even if we were to make the test environments extremely fast and run at almost no load, there will be some amount of time required for DOM tree creation and parsing :)

Hence wait() is a basic required function for Selenium and can not be avoided :(