Automating Web Applications - How I Got Started - 2

If you have'nt read part-1 of the story, please read it so that you will get the full picture.

I appreciate the good design in IBM three tiered architecture, but I believed there could be better object oriented ways. The three tiered architecture gave me some good insights on how to maintain reusability, maintainability and the importance of a good design. It actually inspired me towards a different design.

I imagined a page as an object containing three other objects. componets, actions, and verifications. Each of these will have respective page specific items. Test cases are put in a different layer. A simple test case would look like the following. Think of the below as a psuedo code, just for explaining the concept, not a real one.

Login login = new Login("URL");

login.actions.login(username, password);

if(!login.verifications.verifySuccessfulLogin())

  System.out.println("login no successful");

else

  System.out.println("Successfully logged in as " + user );

I designed this without knowing that there is a similar, but better pattern called pageobject design.

As I told in the previous post, this was a slack-time task, and though I was doing it with a lot of hope and passion, I was the only one doing it. A lot of code representing a lot of pages need to be complete to cover even smoke testing level. But, if the classes are ready, and API is ready, progress from Smoke Testing level to Medium/Thorough testing would be much quicker. The point is, a lot of coding needs to happen to achieve some minimal real progress. No one in the management realized that this is another development project, which demands a lot of programming skills and resources (people). After almost 1.5 years, I still seriously doubt that  they have a good understanding of the load of the task.

After about three months, my supervisor, who is also a very good friend of mine asked a simple question. "Do you think we would be able to make it by the end  of the year if we go in this pace". This let me re-think the whole progress and the target I would achieve at the end of the year.

By that time, only one test case was done. I had managed to ramp up another colleague, the one and only resource I had for sharing the work (only if he was free of other tasks). He was struggling to catch up with Java, and not so enthusiastic because the company is not Java based, rather PHP based. When we got into any Java specific issues, none was there to help. Another major risk was the entire product, the web application, was being re-written using symfony framework at that time. So, the product was changing. One traditional rule in Automation is "Not to Automate Against A Changing Product". The product has to be stable, before starting automation. So, by the time I write another ten more test cases, most probably the entire module would have been re-written making the automation code outdated.

Given all the above facts, I decided this is not the way. This prompted me to go for another solution. I'll explain that in the next post.

Automating Web Applications - How I Got Started

I am in software QA stream. And an Open Source company whose main product was a web application. I was asked to write functional test automation for the web application as the company wants to push in that direction. At the time of the request, i.e. early in 2010, functional test automation was not a high priority, but completing manual testing on new releases had higher priority. Functional test automation was a slack time task, and I was new to the company and web applications.

I had hands on experience in working in a full fledged functional test automation project, trying to automate IBM DataStage using Rational Functional Tester. We were doing automation as per the three tiered architecture. Three tiered architecture separates, appobjects into one layer, tasks into another and test cases into another. Test cases call tasks, and tasks are achieved by manipulating appobjects. Check this link out for more details on the architecture. Though I find the concept not being very much object oriented, still that solution was a good one, scalable and maintainable. So, my whole experience was around IBM DataStage, which is a desktop application.

Now, in the new company, it was about web applications. It is not a web site, to emphasize, but a real desktop like web application. They had done some automation already, which was selenium IDE recordings. I was told that selenium scripts would fail sometimes without proper reasons. I was OK with it because that is something common to any automation suite. But, the degree of emphasize they put on that fact was quite puzzling, and then when I tried things out some other interesting facts also came along. One of the colleagues, who had contributed a lot to the scripts, told me that when the scripts are run in high speed, things often fail. So, running in a slower speed would make many of the test cases pass. I did some googling and could not find such an issue with Selenium. And the scripts did not have verifications built in. Renaming a single texbox in the application would necessitate updating so many places in the scripts.

Slowly the responsibility of automating the entire product came unto my head. My previous experience and expertise taught me that record and play back is not the solution. A definite NO to that direction. Then I had to do some research, to find out a better way to automate the application. The company was willing to invest in commercial products, even though open source technologies are preferred. But, If I pushed them towards big commercial products, I knew there would be too much expectation on me. And if I later find out that we are not going anywhere with that tool, that would impact my career negatively. I wanted to stick to Open Source solutions for so many reasons, including my love towards Open Source. And I am still glad, that I didn't go behind any commercial software.

My research suggested WebDriver would be the best. I started with that, but there were two main problems. I was just a QA guy with some programming knowlege, and WebDriver was just a jar file without any IDEs and frameworks to kickstart. There was no one in the QA team, who could guide me how to proceed in this direction. We had a development team, and I had a good relationship with them, but they don't have the slightest idea of how a functional testing framework would look like. So, I could not talk to them about my needs. As I had been in an IBM Rational Functional Tester project, I knew how a matured functional test project would look like. The pretty fancy test reports, the ant commands that would let us run the test suite with different configurations, the object mapper interface which would give us the properties of the components, and so many other stuff. You can check this link to have a smell of it. All of them are missing, and all I was left with was one lonely jar file and the task of automating the entire product.

My stubbornnes asked me to write all the verifications using if conditions, and some research pointed me that firebug could show me the properties of the components. The dear Eclipse IDE was already there. And I started writing the code for functional testing.

That's how my journey began. Read the next post to see how things moved on.